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SOME CONCEPTS OF JOB FAMILIES AND THEIR 
IMPORTANCE IN PLACEMENT 


HERBERT A. TOOPS? 
Ohio State University 


THERE are, it is asserted, some 30,000 or more occupations. 
That is a large number. We could hardly construct that many 
aptitude tests, say, in acentury. To any other proposal affect- 
ing all the occupations one would have to make a similar com- 
ment. The task is too big; it would not get done. There is 
great need, accordingly, for somehow reducing the number of 
“kinds” of occupations. Could one sort out, for example, a 
small number of type-occupations which would stand for or 
represent the lot of them? 

This hope is analogous with the corresponding dream of 
psychologists regarding human types. They hope to be able 
to type all humanity into a relatively few “unique personality 
profiles,” or patterns. Thus, though the people in a given type 
still would differ considerably amongst themselves, such dif- 
ferences might be thought of as relatively unimportant. The 
people of a given type, however, by definition would be singu- 
larly alike in, say, such matters as aptitudes, health, drives, 
wants, and satisfactions. They might still differ greatly in 
race, color, religion, height, weight, appearance and other 
respects. 

The importance of such maneuvering is easily seen. For 
example, in any psychological, sociological or educational 
study, “like persons” must be subjected to the test of the 
experiment. With only a limited number of types of people 
to be concerned about it would be easy then to get together 


1QOn leave with the National Scientific Roster of Scientific and Specialized 
Personnel. The writer is indebted to Dr. Beatrice J. Dvorak, who made consultation 
with WMC officials possible, and to Dr. Carrol J. Shartle for helpful criticism of the 
article in formulation. 
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enough persons for a study; enough that any statistical follow- 
up would be statistically reliable. Without some such concep- 
tion as “unique personality profiles” every person obviously 
has a profile so “complex” that he is his own type, thus result- 
ing in 140 million such types in America, and no studies are 
possible. 

Just as the botanists find it useful to group their plants for 
study, and the zoologists their animals, so must psychologists 
find a way to group their humans. How to do this is the ques- 
tion. This they may not do by employing the United States 
Census classifications. The psychological differences, for ex- 
ample, are small between whites and Negroes; between men 
and women; or the single and the married. We must search 
deeper for the more fundamental differences among individuals, 
indeed for the traits which are the “basic dimensions” of 
humans. 

The traits on which unique personality profiles are founded 
must be chosen—it is known—to obey such general prin- 
ciples as: 

1. The traits must measure basic human “dimensions” or 
qualities. Another way of saying the same thing is to 
state that: 

a. The traits, in the aggregate, must correlate highly 
with success in all the important endeavors of 
mankind, particularly the occupational success 
criteria. 

b. They must correlate zero, approximately, with one 
another to the end that each such trait thus mea- 
sures as much as may be something different from 
every other, a basic human “dimension.” 

2. The number of such traits to be measured will therefore 
become a-minimum; and the “minimum profile of per- 
sonality” results. 

3. The traits must be objective and quantitative, and 
(preferably) not too expensive of time or resources for 
their accurate measurement; in a word, they must be 
capable of being measured by practical tests or mea- 
suring instruments. 








clas 
pas 
in 

pos 
por 
the 
occ 
to 1 
thi: 


sink 
job 
in | 
pal 


pat 
eve 


Wz 
in) 
pra 
but 
or 

thi 
gui 
im] 


hav 
the 
job 
tio: 
dai 
the 
pre 
wo 














CONCEPT OF JOB FAMILIES 197 


Obviously analogous concepts underlie any attempt to 
classify occupations. The “traits” of occupations likewise must 
pass the same standards. They too must be unique, minimal 
in number, and as objective, quantitative and practical as 
possible. There is here, however, an additional requirement, 
popularly referred to as a means of “bridging the gap between 
the man and the job.” Briefly this means that, if possible, the 
occupation should be measured in the same units as the man, 
to the end that we may form reasonable judgments of whether 
this given man can do this particular job; or rather, since the 
man is the reference point rather than the job—or popularly, 
since industry exists for man and not man for industry—which 
job, of the myriads of jobs, can a man of this particular profile 
in general do best? The reference point often is reversed, and 
particularly in wartime where filling the “openings” is the 
paramount consideration, even though it does not follow in 
every case that the individual “works at his highest skill.” 
War is an emergency; it is met with emergency behavior. Thus 
in wartime we may find it desirable and necessary to employ 
practically all 24-percentile, or higher, chemists as chemists; 
but we also may employ even 100-percentile lawyers as clerks 
or infantrymen. (The professed aim of course is not to do 
this, but, after all, any soldier is one soldier!) With peace, the 
guidance consideration will again assume its former relative 
importance. 

With humans and jobs measured in comparable units, and 
having amassed in addition a very abundant evidence upon 
the efficiency of each given type of man in each given type of 
job, very vigorous modes of sorting men to fill jobs (for selec- 
tion or placement) and of sorting jobs to suit men (for gui- 
dance) at once become possible. The technical problems 
therein are at least partly solved. With the two measurement 
problems solved, the problem of measuring the correspondence 
would readily yield to attack. 

Hitherto this correspondence of job and man has been “de- 
cided”*by a counsellor, employment clerk or placement officer, 
customarily all too ignorant of the varieties of human nature, 
on the one hand, and of the varieties of jobs on the other. And 
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customarily he has thought of the job as a different-order-of- 
creation, with its own traits and categories that bear little or 
no relation to human traits and categories. Clearly then any 
mode of classifying occupations and jobs which will emphasize 
such values and relationships—particularly any which will 
make the relationship obvious—will be valuable and likely will 
lead both to better selection (placement) and to better gui- 
dance. Ideally, if there could be a one-to-one correspondence 
between the occupational “traits” and the man traits, place- 
ment would be maximally facilitated. Possible methods of 
making this correspondence are considered herein. Briefly, 
they are: 

1. Stating the requirements of the occupation in terms of 
the average or, preferably perhaps, the 75-percentile 
as-to-success worker therein, on the traits on which it is 
useful—i.e., valid—to measure the workers. 

2. Specifying an occupation as an idealized set of human 
requirements, or profile of human traits, based on human 
judgments of the occupation’s “requirements.” 

3. Specifying all occupations in terms of the human traits 
which a factor analysis of data, obtained preferably by 
method 1 above, would reveal as the common measures 
of all jobs. 

4. From empirical observations of success on jobs of a very 
large number of men of varying profiles, specify as the 
human-characteristics profile of the job, that human 
profile which succeeds best at the occupation in question. 

The attempt to classify occupations has so far led only 
to the concept of job families. The concept has grown out of 
employment office work, where in normal times they have on 
hand unemployed men needing work. Their search at such 
times is for an available job which a given man, say an account- 
ant, can do. Their special knowledge consists in convictions 
of what a given type of man can do. 

Our problem is to increase that knowledge and to amplify 
its dependability. During the depression it became obvious 
that an accountant could do labor work, if he had to, in order 
to live. Even depressions, however, do not greatly increase our 
knowledge of what more complex work accountants can do. 
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Clearly, if we could systematically accumulate such experi- 
ence for all or most men of the community as they change jobs, 
we should soon have a vast fund of knowledge for sifting and 
refinement. In wartime many drafted men arbitrarily and by 
chance are placed in occupations new to them, but in most cases 
these too are down-grading; and furthermore, little record is 
made of the satisfactoriness of such placements. At such times 
much job-dilution (job-simplification) is resorted to in an 
effort to employ less capable talent on small units or special- 
ized aspects of more complicated work, to improve production 
and to increase accuracy by the development of great individual 
skill on non-complex tasks. 

It has long been known in employment office work that 
when industry could not get “exactly the type of man it 
wanted,” a man of “certain specific occupations” often might 
do almost as well. In Swan’s Index, for example, prepared for 
the C.C.P. personnel work of World War I, the “substitute 
occupations, or civilian equivalents,” of some 750 Army trades 
and occupations were determined and listed for the use of all 
personnel officers. The problem, it has been stated, is that of 
how to transfer workers from job to job with the fullest pos- 
sible utilization of their previous skills. Thus the Army should 
exploit the civilian occupational skills of the draftee, and in 
turn industry, after peace, should exploit the Army-acquired 
occupational skills of the demobilizee. Inasmuch as any defi- 
nition is always defective, let us enumerate the possible char- 
acteristics of a “job family,” by implication, from a considera- 
tion of the actual or proposed practical uses of such “job 
families”: 

1. When up-grading, training will be more effective if it is 
for a new responsibility which falls in the same family of occu- 
pations. Thus shortages of manpower may be readily filled 
from excesses of manpower if such exist anywhere in the same 
family of occupations. By such action the amount of re-train- 
ing will be minimized. (It will be recalled that different occu- 
pations turn out different products and these are peculiarly 
affected, in wartime particularly, by the presence or absence 
of raw materials. Accordingly, “excess” of manpower may 
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crop up almost anywhere after every new governmental order 
or decree. ) 

2. For determining substitute occupations for individual em- 
ployment purposes, where the additional training, if any, is to 
be afforded by the job itself, job families provide the necessary 
factual information. The greater job families, based on indus- 
tries rather than occupations, enable wise decisions to be made 
in converting non-essential industries to essential industries, 
with a minimum of waste.* The complete present manning 
tables of a factory compared with the proposed manning tables 
of a new production schedule reveal at once the distribution 
of occupations freed and those necessary for operations after 
the conversion. Let the distribution of occupations released be 
abscissae (X) of a two-way table; let the proposed new distri- 
bution be ordinates (Y) of the same table. The conversion 
is effected then in the following series of steps: 

2(1). Into the diagonals of the table, which have identical 
labels for each coordinate, are placed the several frequencies 
of persons who can be transferred directly without conversion, 
into exactly the same occupations. The personnel cards, or 
duplicates thereof, of these specific persons are then sorted from 
the total work-roster and are filed under the new manning 
categories. 

2(2). In each diagonal compartment, form a ratio of the 
number thus obtained to the total number desired in each of 
the several occupations. Parallel this record with another of 
arbitrary indices of indispensability of the unfilled portions. 
By a study of these two ratios, giving weight to the importance 
of the new occupations in the scheme of production, and to 
salaries, training time, and possibility of obtaining outside 
trained recruits, establish an order for filling the new occupa- 
tions. It would be highly desirable if this angle of the matter 
could be made objective and mathematically determinate. 

2(3). Reduce the X-marginal frequency, or abscissal mar- 
ginal row of the chart, by the frequencies of persons just allo- 
cated (Step 2(1) without conversion). 


2 Shartle, C. L., Dvorak, Beatrice J. and Associates. “Occupational Analysis 
Activities in the War Manpower Commission.” Psychological Bulletin, Vol. XL 
(1943), 703. Shartle, C. L. et al. “Ten Years of Occupational Research.” Voca- 
tional Guidance Magazine, XXII (1944), 387-448. 
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2(4). Considering now the new occupation (Y) to be first 
brought up to strength: by aid of the job family classification 
pick out the first conversions therein, if any, those which in 
general can be made with a minimum of retraining. Add the 
frequencies of the corresponding persons to the compartment 
which is designated by an X of the to-be-invaded occupation 
and a Y of the to-be-established occupation, somewhere be- 
neath the diagonal row of compartments; and remove a corre- 
sponding number from the X-marginal frequencies and also 
from the remaining roster the cards of those so related and 
converted. Similarly invade the other earliest-to-be-invaded 
occupations; and when all is complete establish in a second 
Y-marginal column the frequency of the most indispensable of 
the new occupations. If the quota is filled, the process is 
stopped short at that point, only enough persons being taken 
from the last occupation to fill the need. 

2(5). In similar fashion, consider the second occupation 
to be brought up to strength, the third, and so on until all the 
possibilities of first conversions are exhausted, or until the 
quota is filled, whereupon the sorting of cards is at once 
stopped. But if this does not yield enough personnel, resort 
next to the second conversions, those requiring more training, 
and to the third and higher conversions, if any, until either 
the quota is filled or there are no more possibilities of conver- 
sion remaining. The several conversions may be kept separate 
by different colors of ink, and are a training order in detail to 
the training department: “You are to train so many lathe- 
hands to be assemblers, etc.” 

2(6). Consider next the second most urgent new occupa- 
tion, repeating steps 2(4) and 2(5). 

2(7). So continue until there are no more conversions to 
be considered. 

There will now be two groups of interest: 

A. A residue of personnel not useful to convert. If there 
are shortages in other areas of production, these may be 
subjected again to a second scanning with the end of filling 
these shortages. 
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B. Shortages in the proposed new manning tables which 
must be filled by resorting to outside recruitment. 

Clearly there is room here for the development of mathe- 
matical methods of optimalizing placements. 

3. Transferring an unsuccessful person within a firm into 
an occupation in a job family in which he had formerly been 
successful, as an alternative to separation from the firm, is wise 
personnel management. 

4. In giving guidance to individuals who wish an improve- 
ment in their situation but are unable to forego an income while 
retraining, job families are helpful. 

It is a well-known principle of adult guidance, for example, 
that if possible the guidee should exploit his most successful 
experiences and training rather than merely abandon them 
when he changes occupation. 

5. In recruitment of personnel for new industries, job 
families inventories yield valuable information as to where 
such recruitment will yield the best results at the least cost. 

6. In the assignment of recruits to related military occu- 
pations, job families prepared for the Army, Navy, Marines 
and U. S. Coast Guard have been helpful.* 

7. That wages of the member occupations of job families, 
other things being equal, should differ but little, is a principle 
of value in wage adjustment. 

8. The various occupations of job families probably have 
highly similar physical demands, thus making it possible to 
know alternative positions into which a given handicapped 
person will fit, without preliminary research or trial and error. 

9. In promotions one wants to maintain the principle of 
individual growth, rather than that of revolution, so that a 
knowledge of job families should be helpful in establishing 
paths of promotion. One is ready for promotion when one is 
proficient in his present job in all the elements thereof that are 
common to the next job ahead, this assuming that the paths 
of promotion previously have been established according 
to such principles. One’s present job is thus always a produc- 
tion field for exercise of the skills, knowledges and tech- 


3 Shartle, C. L. et al., op. cit., p. 704. 
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niques obtained in the previous position, and a training field 
for acquiring those skills, knowledges and techniques basic to 
the production of the job ahead which also are a part of the 
reasonable practice of the present position. 

10. In aptitude test construction, a test conceivably may 
be devised which will be reasonably adequate for all the occu- 
pations of a job family, thus restricting and constricting the 
field a great deal. 

11. The reabsorption of veterans and the shift of civilians 
from wartime production to peacetime production after peace 
and the demobilization will involve a stupendous task of 
worker transfer, retaining and allocation. As an aid in this 
task, job families, applied in reverse, will be useful to locate 
the logical job destination of workers in civilian industry when 
war and war industry no longer exist. In theory this is analo- 
gous with the construction of a decoding code. Inasmuch as 
it is reasonable to believe that many demobilized at peace will 
be ready for promotions, their after-war destiny logically also 
should be a related job-family occupation to which a given 
worker “should be promoted” in view of his increased “skill” 
acquired during the war years. The longer the war, the less 
advisable it is for a larger and larger number of the de- 
mobilizees to return to their former jobs. 

The common statistical element in all the above uses of job 
families is that a few “type” jobs may stand for all jobs just 
as a few personality profiles may stand for all human profiles. 
The end to be secured is a great reduction in the magnitude of 
the problem and in the amount of data or evidence which must 
be obtained to solve a problem. 

The War Manpower Commission says that there are six 
classes of requirements* on which occupations should be com- 
pared, namely, 

(1) Nature of the work done. 

(2) Tools, machines and other aids employed. 

(3) Materials worked upon. 

(4) Traits required of the worker. 


(5) Knowledge (including specialized atest required. 
__ (6) Experience. 


# Stead, W. H. and Shartle, C. L., op. cit. 
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Occupations grouped on the basis of the work performed 
(i.e., sorted on the basis of action verbs, such as welding, sew- 
ing, nailing, gluing, etc.) seem more alike than if grouped on 
any other basis. In the United States Employment Service 
sorting of the “traits of occupations” to determine the job- 
families, the first four of the above list occur in the order above 
named in the sorting operation. Shartle’s results seem to show 
that “people,” rather than specific kinds of people, are required 
for a majority of the occupations of industry, and that the 
training time is, for the great majority, unbelievably short. 
Wartime training methods of unusual potency emphasize the 
correctness of this conclusion. Motivation was often at a 
maximum. If you did not learn, you got your head shot off! 

As a means of facilitating their use, the coding scheme 
employed to stand for an occupation should take into account 
the family relationship of occupations by ascribing contiguous 
numbers to the highly related occupations. Occupations thus 
break down into “fields of work,” somewhat narrower “process 
groups” and, finally, into still further variations, varieties, 
alphabetically arranged (and distinguished by unit digits in 
the USES code number). 

Just as in the taxonomy of botany one may arrive at various 
conceptions of what is a “family” (a sub-division of higher 
divisions of classification) so one may arrive, by different 
routes, at different classification principles, at different aggre- 
gations of occupations which in the several classification 
systems logically may be called families. Some half dozen 
such alternative, actual or potential systems of deriving fami- 
lies of occupations will now be outlined. 

1. The most extensive development of job families has been 
made by the technique developed by the WMC, Worker Anal- 
ysis Section. It involved the following steps: 

1(1). A job characteristics form, consisting in its final edi- 
tion of 47 human traits, was filled out by from one to fifteen 
analysts in different parts of the country observing the 
same occupation, as distinguished from job-in-this-particular- 
factory. 

1(2). The same analysts also observed the duties of the 
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worker, the tools used, the equipment, the materials and the 
minimum education and experience necessary for the work. 
The results were systematically recorded in a Job Analysis 
schedule form. 

1(3). A compromise set of “traits” for the given occupa- 
tion was decided upon, and was entered into a Master Worker 
Characteristics Sheet. 

1(4). The results now were recorded into Speedsort cards 
and other pertinent observations about the occupation were 
written on the face of the card. 

The cards now represented in highly condensed form the 
results of judgments of what are the important characteristics 
of the occupation in question. Some 9,000 occupations, in some 
85 industries, were thus reduced to Speedsort cards, a kind of 
catalog of occupations which could ultimately, after the job 
is finished, become the card catalog of American occupations. 
By the aid of the usual sorting needles, presumable families, 
having regard for the above list® of judged most-important 
“traits,” were sorted out. This was aided by noting that as the 
sorting progressed, channels in the edges of the cards began 
to develop on other traits of a family than in those traits actu- 
ally sorted upon, the traits of the list above. Aided by these, 
the member-occupations of presumable job families were 
sorted out. 

The complete job analysis description was appealed to as 
a further test for excluding some of the presumable members 
of the family; and to insure that all that should belong to a 
family had been included. The job family was now complete. 

1(5). There was still a need for ascertaining which of the 
members of the job family were most like the ideal “job type” 
from which all the occupations of a family deviate in some 
respects, even “important” ones. By a highly subjective pro- 
cedure, based first on a guess as to the relative importance 
(weights adding up to 100) of the half-dozen or more general- 
ized aspects’ of the type occupation, and second, upon the 
subjective points-rating therein of a given occupation, never 





5 The aspects as above outlined are: (1) work done; (2) tools, machines and 
other work aids used; (3) materials used; (4) worker characteristics required; (5) 
experience required; (6) training (including special training) required. 
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exceeding the previous “weight” of this particular member- 
occupation of the job family, there was ascertained an aggre- 
gate numerical value (the sum of the points-ratings) by means 
of which the specific occupations could be ranked in a decreas- 
ing order of degree of resemblance to the type job. 

By aid of these, the job-family was split into three sub- 
divisions: 

Closely related occupations—those in which workers of 
experience could probably be transferred to any other 
member-occupation of the closely related group with little 
or no retraining. 

Less closely related occupations—those for which 
workers required more retraining to make them acceptable 
workers. 

Least closely related occupations—those barely meeting 
the minimum requirements for useful similarity, and for 
which workers, although they might require complete re- 
training, nevertheless because of their worker character- 
istics and other considerations of knowledge, skills, etc., 
would be more likely to succeed at the work than just any- 
one chosen at random. 

The apparent strong points of the method are: 

a. A wide variety of “traits” of occupations was in- 
vestigated. 

b. The analyses of the job were made by trained ana- 
lysts. 

c. Observations of the same occupation were made in 
different parts of the country so that sectional dis- 
crepancies could be allowed for, even to the extent 
of splitting up an ostensible occupation into two or 
more; or it could be noted that occupations bearing 
differing names in different parts of the country are 
in reality one occupation, not several. 

d. A common denominator of the several alternative 
analyses was decided upon. 

e. The job families resulting had other than statistical 
and logical definition and justification, since the sort- 
ing of the Speedsort cards was augmented by a 
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critical scanning of the entire description of the con- 
stituent occupations before adoption. 


Its more obvious weaknesses are: 


a. 


b. 


The analyses were subjective, even though done 
after a uniform schedule or outline. 

The list of job “traits” was not wide enough to cover 
adequately the professions. 

The comparison of the reports of the as many as 
fifteen analyses of a given occupation was done 
centrally by a person out of touch with the field and 
without giving the analysts who did the work a 
chance to correct the most obvious misconceptions. 
The completed analysis was not generally sent to 
them for criticism before official adoption, although 
some one or more analysts scanned each of the pro- 
posed analyses before adoption. 

Different Speedsort operators, with the same end 
in view, presumably could come up with different 
potential job family-occupations. In other words, 
the system was not fully statistically determinate. 
There is some doubt whether the skills and traits 
of workers are in every instance transferable merely 
because the pattern in the Speedsort cards is highly 
similar. A watch-maker and a cannon-barrel borer 
might come out in the same job family and on the 
basis of the statistics have every warrant for be- 
longing to the same family, yet the psychological 
characteristics, particularly as to precision, may be 
quite different so that actually there is little trans- 
ferability of skills. 

The double dose of subjectivity involved in putting 
the sub-members of a job family in decreasing order 
of resemblance to the common type is unfortunate 
when there exist several alternative techniques, 
independent of subjectivity, which will give a nu- 
merical measure of the degree of correspondence of 
two profiles. The ideal jobs or type jobs we take to 
be a pure figment of the imagination. It has no 
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more objectivity than anyone’s conception of the 
“ideal mammal,” the “ideal liliacea” or any other 
taxonomic classification. In fact it does not have 
the rigidity of definition of a sub-classification of 
botany in that ail liliacea are spermatophyta (seed- 
bearing, with true roots, stems and leaves), angio- 
sperms (with true flowers, each containing, nor- 
mally, four whorls of floral organs), autophytic 
(possessed of chlorophyll), monocotyledonous (one 
cotyledon in each seed and with parallel-veined 
leaves) and bulbous (having bulbs). 

2. Inverse factor analysis may be employed as an alterna- 
tive means of ascertaining a job family. If the characteristics 
of an occupation can be quantified and measured, a factor 
analysis—occupations replacing the usual human names—will 
reveal what factors of job-traits underlie occupations in general. 
In this there is no necessary restriction of “the occupation” to 
any particular pattern of measurements, for these may be quite 
as broad as, or broader, than the WMC list above; and in the 
case of the professions® almost surely will be broader than that 
list! Occupations having similar factor loadings belong to the 
same family. 

3. A variant on method 2 above emphasizes successful 
plying of an occupation and implicitly recognizes that every 
occupation has at least a fringe of people “who ought not to be 
in the occupation.” If the various X’s of the above method 
are replaced with a sufficiently varied list of the human traits 
of, say, “successful workers” of the occupations in question, 
the resulting factors will discover the basic human factors of 
workers. In this case also one may express the occupation in 
terms of its component factors and factor loadings. The load- 
ings, here, are the profile of the occupation; and those occupa- 
tions with highly similar profiles are jobs of highly similar 
human requirements. The occupations which resemble each 
other most comprise the job families. 


6 Professions are generally characterized by a higher level of job-traits than 
characterizes occupations which are not professions; in particular in such respects as 
research, administration and supervision, and knowledge. There is probably no 
job-trait found uniquely among professions save that of being dubbed a “professional.” 
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The occupations of a given job family presumably may be 
located after the occupational profile is obtained by such 
methods as 1 or 2, either by (1) the Sagebeer Index, or alter- 
natively by (2) the Stephenson inverse factor method. In 
employing the Sagebeer Index, a rationale of research for 
locating families would need to be developed. 

4. A fourth method might be dubbed the universal follow- 
up method. If one kept track of all or nearly all placements 
of professional personnel and at the conclusion (resignation, 
discharge, transfer) of the jobs of such personnel collected a 
simple verdict as to whether or not the person in question had 
been “satisfactory” in the position in question, one might 
analyze the resulting data to note by a simple correlation index 
which occupational transfers are successful and which are not. 
With all the occupations of concern appearing in alphabetical 
order as both the ordinates and the abscissae of a two-way 
table, one may let Y be prior job and X be subsequent job. 
In the compartments thereof, let four-fold correlation coefh- 
cients, solved by the formula, 

oy ad—be 
V(a+e) (b+d) (c+d) (a+b) 
be recorded. (See Fig. 1 below.) 
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Figure | 
Four-fold Validity Plot of Success on Prior Job and on Subsequent Job. 


(If success can be more adequately measured than implied in the bifurcation 
illustrated, this four-fold plot would be replaced by an m X m-fold plot) 
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The letters of the diagram (Fig. 1) are the frequencies of 
the table, here recorded literally to show the mode of compu- 
tation by the formula. From such a table of r-indices, by 
means of a suitable index to be devised, one could readily sort 
out the job families. It will be noted that one obtains here 
two-directional indices. Thus, not only would one consider 
the question “Are the failures of job A also failures when they 
subsequently enter B?” but also “Are the failures of B failures 
when they subsequently enter A?” If the two tendencies are 
vastly different, as in some cases they may be, we might dis- 
cover here some preferred sequences of progression in acquiring 
vocational skills. 

It is normal for wide-awake, alert human beings to take on 
more and more skills, including jobs and even occupations, 
as they develop and mature. We know little enough about the 
more complex end patterns characteristic of “the expert.” 
One interesting question here is whether the expert usually is 
characterized by having acquired his complement of skills in 
an orderly progression. If the answer were in the affirmative, 
obviously one might similarly order the acquirement of skills 
of all people to very good advantage. This method logically 
requires that all the experience of a large community of occu- 
pations be amalgamated, say of a WMC area at least. 

5. The logical or evolutionary concept of job families, de- 
scribed by Knott,’ conceives that jobs which belong to a family 
are the variations which a specialized industry has evolved out 
of a simpler craft. Thus hat-making, cap-making, dressmaking 
and even baggage-making in common employ variants of the 
power sewing machine (both the tool and the corresponding 
skills), itself an invention superseding in turn hand-sewing. 
And typists, stenographers and secretaries are variants of the 
amanuensis or letter-writer who still plies his skills in countries 
where most of the inhabitants are illiterate. In such variants 
some retraining is needed before an individual successful in one 
mode of sewing may become proficient in another, but, obvi- 
ously, the retraining period would likely be much less than if 


7 Knott, Edward E. “Job and Occupational Analyses.” Missouri State Employ- 
ment Service, April, 1941, p. 47. 
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a highly skilled operator in any one of such were to be trained 
into a highly skilled operator of, in general, any other job not 
included in the job family, say a lathe operator. Thus an index 
of what the educational psychologists call “transfer value” 
would seem to be at the heart of this concept, which is based 
on logical, that is economic, historic, or evolutionary, concepts 
of the mode of development of the occupation in question. 
Insofar as such families can be ascertained without the neces- 
sity of the empirical evidence of the #4 method above— 
empirical ascertainment of transferabilitys—the method serves 
usefully to restrict the field of inquiry but does not eliminate 
the need for the previously mentioned modes of study. There 
are sub-families within families and sub-sub-families within 
sub-families. These may be ascertained more surely by sta- 
tistical analysis than by unaided observation. One strength 
of the method is that more than the one source of data, the 
customary field observation of jobs by trained workers, is 
insisted upon as necessary to a proper classification. (To 
properly classify plants, one must study paleobotany as well 
as botany.) With a set-up such as No. 1 or No. 2 above, the 
sub-occupations, members of eventual potential sub-family 
groupings, would be the Y’s of a basic data table whereas the 
X’s might be either (a) tools employed, (b) unit-operations 
employed, (c) motions, in the Gilbreth sense, employed, or 
(d) worker characteristics, or any compound of the several 
kinds of such components, or (e) any other of the WMC classes 
of observations. The subsequent analysis of the collected data 
would follow that of Nos. 1 and 2 above. 

6. An alternative to all the preceding is that developed by 
Mr. Robert Shosteck and associates of the Roster of Scientific 
and Specialized Personnel, to meet the emergency in prospect 
in the sciences when demobilization occurs. It is simply that 
of writing to industrial experts employed in practical work, in- 


cluding the selection of scientific and specialized personnel, and 


8 Evidently the indices of success of the basic table of the No. 4 method might be 
replaced by indices of amount of trainability required, say the average of retraining 
actually administered, it being assumed that retraining is stopped as soon as pro- 
ficiency is achieved. 
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asking them to give their judgment as to the degree of trans- 
ferability—on a scale: 


1. Considerable (Less than six months additional train- 
transferability ing required ) 

2. Moderate (Six to twelve months additional train- 
transferability ing required ) 

3. Slight (Over twelve months additional train- 
transferability ing required ) 

4. No (Transferability unfeasible, since little 
transferability or no training time would be saved ) 


—of people in a given series of occupational specialties of, say, 
civil engineer, when and if these were to enter the break-down 
specialties of the same occupation or the specialties of another 
industry, say the engineering specialties of municipalities. The 
present hypothetical occupations of the to-be-released workers 
are Y-ordinates of a two-way table, while the X-abscissae are 
the jobs of an engineering sort now maintained in municipali- 
ties. In this case, hydraulic engineers, for example, were asked 
to judge on the above scale the transferability of each of a 
number of kinds of civil engineers to the several other special- 
ties of the same occupation. The validity of the method hinges 
on the ability-as-judges of the addressees, i.e., upon such factors 
as the extensiveness of their actual knowledge and experience 
with the implied situations judged, the criticalness of their 
judgment, and their willingness to take the necessary time— 
all of which could no doubt be brought to a state of excellence 
by well-known techniques, such as that of using lesser experts 
to pick the greater, and the like. The method has the merit 
of quickness, low cost, and independent verification of the 
observations exploited. It has all the weaknesses of using 
untrained judges in what is essentially a psychological experi- 
ment. It may be pointed out that the WMC job analysts also 
employed judgment, of small elements of an occupation to 
establish a profile from which, by statistical manipulation, the 
job family ultimately was determined and the transferability 
of skills was inferred. Here, however, the transferability of 
the specialty as a whole is judged. By abstract psychological 
principles the judgment required here is easier to obtain but 
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is of lesser validity than judgment of the more specialized 
sort obtained in the former studies. For the aid of demobili- 
zation counsellors, the less costly method yields results which 
presumably are about as fine as demanded for the purpose. 
The finer break-down of the WMC job analysis obviously en- 
ables more uses to be made of the results, such as advising the 
demobilizee to know what specific education or training he 
should acquire for a better all-round fitness in his new vocation. 
This the Roster technique, with its present paucity of data on 
the individual profession, cannot supply. However, having 
helped the demobilizee to choose wisely a new occupation 
which will exploit his army and pre-army training, such addi- 
tional (educational) information in many cases may be had 
even more usefully from a new counsellor, perhaps a dean or 
secretary of an engineering college. The advisee is less con- 
demned by receiving only parts of the truth than by receiving 
mistruths. If the validity of the method is equal to or only 
slightly inferior to the more laborious method, obviously then, 
on this score alone, it has great merit. Incidentally the statis- 
tics of ratings, for subsequent treatment of the results, are 
already well worked out. The necessary statistical indices in 
no case are an insuperable element of the whole. The rating 
scheme for rating officers’ traits in World War I was deemed 
a rating failure by its chief statistician.° 

Confronted with such a variety of actual or potential tech- 
niques, one feels a need to inquire, “Which is best?” The 
question implies the existence of some standard, some criterion, 
by means of which one might adjudge relative merit. Insofar 
as any such system is, from one angle, merely a filing or classi- 
fying system, obviously all the systems solve at least minimally 
the problem in the sense that every occupation has a position, 
and a code number, and none is omitted. However, a classi- 
fication system may do and usually does do more: it leaves gaps 
to be filled in by variant and newly emergent classes. For some 
purposes the gaps may be quite as important as the filled 
niches. 


® Rugg, H. O. “Is the Rating of Human Character Practicable?” Journal of 
Educational Psychology, Vol. XII (1921), 425-438; 485-501; et al. 
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The common purpose of job families, if the multitude of 
purposes enumerated above can be subsumed under one gen- 
eral statement, is that of ascertaining the paths of transfer- 
ability of occupations in accordance with the need for the 
minimization of retraining. It is by this criterion then, if we 
are willing to accept it, that the competing methods must be 
judged. 

The above statement implies a mathematical function 
which may be minimized (or maximized, according to its set- 
up), a matter which is at once both the delight and the despair 
of the statistician. Suffice it to say, at this stage, that the best 
elements of all the methods likely will yield a better result than 
blind adherence to any one alone. In addition it must be 
pointed out that the statistically most elaborate system, the 
WMC system, was worked out primarily for the list of occupa- 
tions job-analyzed in the course of the field observations which 
were paralleled by the development of the Occupational Dic- 
tionary. Only a very few professions were so analyzed. The 
implications of this will need to be thought through in arriving 
at a conclusion as to the potential values of this interesting 
new “measurement” device. Only a complete analysis of the 
entire problem, including the method of “bridging the gap 
between man and job”” (the coding, Hollerith sorting, and 
placement philosophy and practice) will reveal what elements, 
if any, are worthy of adoption. This statement would empha- 
size highly": also capacities and training (or education) to the 
point of specially mentioning them as well as skills. 

It has been suggested that the study of occupations is a new 
social science.* Statisticians have long known that occupation, 
correctly ascertained—which the United States Census does 
not even pretend to do—is probably the most basic classifica- 
tion of mankind; of greater human concern, generally speaking, 
than race, color, sex, marital condition, education, age or place 

10 One simple solution to this problem is to say, “This job requires a man to 
grind valves; this man can grind valves.” 

11 Shartle, C. L. et al. “Establishing Families of Occupations.’ 
Guidance Magazine, Vol. XXI (1944), 405-414. 


12 Kitson, H. D. “Occupationology—A New Science.” Occupations, XXII 
(1944), 447-448. 
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of residence. These categories have the force of tradition be- 
hind them; but they are not as basic, psychologically, sociologi- 
cally or economically as is vocation. Economically all other 
human values directly or indirectly stem out of or are obligated 
to vocation. In correctly classifying occupations, then, we are 
making possible not only accurate studies of this all-important 
institution but also are laying the foundation for gaining a 
control of such matters as guidance, promotion, transfer, and 
the conservation of human talent generally. A statement 
about one aspect of the matter will make clear the importance 
of the matter. If a worker before changing his occupation 
knew with a high degree of certainty whether a proposed 
change likely (i.e., on the average) would result in greater 
opportunity for him, or the converse, much human loss and 
unhappiness could be prevented. Again, assuming a compa- 
rable development in human traits, if one knew what was the 
actual distribution of wages in two occupations for people of 
identical trait profiles, the present unjustified discrepancies 
would tend, without artificial restraints, shortly to disappear. 
And if, finally, a similar distribution of happiness-scores (on 
a test of satisfactions, often called morale in the Army and 
industry) could also be attached to the individual profiles in 
the above table any individual knowing his own profile could 
choose his occupation so as to meet his need, whether for a 
greater income, greater happiness or satisfaction, or both. 

The lower the prestige of one’s occupation, in a scale of 
prestige values, the more a man expects, normally, to get his 
satisfactions outside his work. In the professions one’s work 
frequently is both one’s vocation and one’s avocation and 
recreation. To maximize the individual’s salary and job- 
originated satisfactions is one of the crucial steps in the maxi- 
mization of manpower. 
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TESTING FOR ADMINISTRATIVE AND SUPER- 
VISORY POSITIONS 


MILTON MANDELL 
United States Civil Service Commission, Washington, D. C. 


Tue field of testing for administrative and supervisory posi- 
tions is one which is approached by many psychometricians 
with a feeling of defeatism. A number of studies have been 
made to determine which testing methods will improve the 
selection of administrators and supervisors but, in general, 
psychometricians have tended to stay away from this phase 
of testing. The reasons for this situation are well summarized 
in Dr. L. L. Thurstone’s statement: 

The intellectual and temperamental qualities that insure 
success in administrative work are probably more complex 
than almost any other group of abilities that can be thought 
of. Psychologists who investigate fundamental human traits 


would undoubtedly seek to investigate first those traits which 
can be assumed to be less complex. 


Although the complexity of the task of experimenting in this 
field is recognized, there still exists in industry, in the Army 
and Navy, and in government, the problem of selecting persons 
for supervisory and administrative positions. With a labor 
force, both civilian and military, of about 60,000,000 in the 
U. S., one can roughly estimate the existence of about 2,000,000 
supervisory or administrative positions. There has been little 
study of the validity of present selection methods for these 
positions, but the great interest of high administrative officials 
in improved selection devices is some indication of the lack of 
values in techniques in use at the present time. 

The following are general definitions of supervisory and 
administrative positions. By a supervisory position is meant 


1L. L. Thurstone. A Factorial Study of Perception. University of Chicago 
Press, 1944. 
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one which involves responsibility for the working conduct of, 
and the quality and quantity of work produced by, one or more [ 
subordinates. By an administrative position is meant one | 
which involves extensive responsibilities for planning, organ- | 
izing, directing, stafing, budgeting, and coordinating the work | 
of an organization or part of an organization. The adminis- 
trator in some cases performs no supervisory duties since he 
may act only in an advisory capacity, but he usually has the 
functions of a supervisor in addition to his administrative 
duties. 

Supervisory positions can generally be classified on the basis 
of three factors: (a) the number of employees supervised, (b) 
the nature of the work or the occupation supervised, and (c) 
the supervisor’s level in the organization. It is reasonable to 
present the hypothesis that the skills and abilities required to 
supervise a few employees are different in degree, and perhaps 
in kind, from those required to supervise hundreds of em- 
ployees. For example, the supervisor who likes to check care- 
fully and in detail the work of a few subordinates may be 
successful, but this same action would probably make him a 
bottle-neck if he applied the same method to the supervision 
of a hundred employees. Observation indicates that this 
method is as much a trait of the individual as it is a function 
of the position that he occupies. 

The nature of the work or occupation supervised is an 
important distinction between supervisory positions. The 
methods required to supervise a gang of ditch diggers success- 
fully differ from the methods required to supervise a staff of | 
skilled psychometricians. Some supervisors who have trans- 
ferred from organizations where strict disciplinary methods | 
were in use have failed miserably in attempting to apply the | 
same methods on different occupational groups or in different 
work situations. 

The trend in the field of administration has been to con- 
sider administration as an occupation per se and to emphasize 
the common elements among such positions. This trend has 
been especially strong in the field of public administration, it 
is noticeable in Army and Navy administration, and it is evi- 
dent in the field of business administration. 
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Those opposing this trend claim that knowledge of the 
business or subject-matter of the organization that is ad- 
ministered is also important. Agreeing for the moment that 
administrative skill is more important than subject-matter 
knowledge, however, one can still classify administrative jobs 
into a number of categories on the basis of other considera- 
tions. These categories can be considered to be: (a) positions 
whose problems are of an international or national rather than 
local character; (b) positions involving the “selling” of the 
goals and objectives of the organization to an indifferent or 
even hostile clientele; (c) positions of an advisory rather than 
operating character; (d) positions requiring the ability to ad- 
minister large and complex functions; (e) positions involving 
swiftly moving problems as contrasted with situations where 
speed is not so essential; and (f) positions involving the ad- 
ministration of a large organization with hundreds of thousands 
of employees as distinguished from a small organization with a 
few hundred employees.’ 

The above categories have been presented on the assump- 
tion that some of the skills and abilities necessary for successful 
performance vary according to these categories. Actually, 
these classifications are based on job analysis and it may be 
found that, in terms of testing, other categories may appear to 
be more significant.* 


Qualifications 


Before discussing actual results obtained in testing for 
supervisory and administrative positions, it may be worth while 
to present various opinions on the qualities necessary for suc- 
cessful performance in these positions. Dr. W. V. Bingham 
lists a large number of such qualities: the administrator doesn’t 





2 These differences among supervisory and administrative positions, which are in- 
tended to be illustrative, indicate that thorough job analysis is extremely essential in 
testing work in this area and that the psychometrician should probably have the 
assistance of a person trained in administration to help him identify these differences. 

8Cf. Alexander H. Leighton. The Governing of Men. Princeton: Princeton 
University Press, 1945. Leighton, who is a psychiatrist, in his study of a War Reloca- 
tion Authority center, divided administrators into two groups: “people-minded” 
versus “stereotype-minded.” His basic distinction between these two groups is based 
on whether the administrator puts emphasis on human values or on adherence to 
regulations. 
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go off “half-cocked,” makes consistent decisions, has well- 
thought out policies, obtains real staff participation in the 
formulation of policies, builds a team, has ideas, listens to new 
ideas, is not an egotist, gives recognition for good work, dele- 
gates responsibility, sees his staff, has good timing in making 
decisions, and knows outside conditions.* Johnson O’Connor 
reports that successful executives have a large vocabulary, a 
wide range of aptitudes, an objective personality, an accounting 
aptitude, and an aptitude for their first position.® 

Schell lists as the outstanding requirements for an execu- 
tive an innate interest in and affection for people, an outstand- 
ing personality, and a scientific mind. Ordway Tead, who has 
contributed extensively to the theory and practice of adminis- 
tration, has suggested the following qualities as desirable for 
leaders: physical and nervous energy, a sense of purpose and 
direction, enthusiasm, friendliness and affection, integrity, tech- 
nical mastery, decisiveness, intelligence in many directions, 
and teaching skill... In a pamphlet published in 1923, the 
American Management Association recommended the follow- 
ing qualifications and values in selecting supervisors: per- 
sonal—30%; mental—45%; moral—l15%; and _ physical— 
10%.* Cleeton and Mason offer as their criterion of executive 
ability “above average ability in a large number of qualities 
which can be rated or measured.” 

It is apparent from this listing of qualities needed by ad- 
ministrators and supervisors that major stress is placed on 
aspects of personality. In addition, however, a number of other 
abilities are considered essential. While few would require that 
the administrator or supervisor should have the highest mental 
ability within the group that he heads, superior mental ability 

4W. V. Bingham. Administrative Ability. Washington, D. C.: Society for 
Personnel Administration, April, 1939. 

5 Johnson O’Connor. Characteristics of Successful Executives. Hoboken: Stevens 
Institute of Technology, 1932. 

_ Erwin H. Schell. The Technique of Executive Control. New York: McGraw- 
a Tead. The Art of Leadership. New York: Whittlesey House, 1935. 
pee 8 ——s the Supervisory Forces. New York: American Management Associa- 


®Glen U. Cleeton and Charles W. Mason. Executive Ability: Its Discovery 
and Development. Yellow Springs: Antioch Press, 1934, p. 12. 
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is still considered very important.’® Part of the differences of 
opinion on this subject may be accounted for by the failure to 
relate the mental ability required to the group being super- 
vised. The mental ability required of a foreman of laborers 
is obviously lower than that required of a Construction Super- 
intendent but in relation to the mental ability of his laborers, 
the foreman should rank high.” 

The administrator and supervisor must be able to under- 
stand organization. Some technically minded persons are con- 
stantly frustrated by administrative procedures and regula- 
tions when they are promoted to supervisory positions, while 
others acquire a knowledge of how to get things done which is 
basic to success, especially in a large organization. Without 
some evidence it is impossible to state what factors make for 
success in this particular phase of administrative work, yet 
one can speculate that this ability is non-intellectual but re- 
lated to the interests and personality of the person.” 


Use of Paper-and-Pencil Tests 


1. Interest Inventories. From the published studies, there 
is a definite indication that the measurement of interests can 
contribute substantially to the selection of administrative and 
supervisory personnel.'* The major problem is in determining 
what norms to use and which special interests, in addition to 
social interest or interest in people, are desirable for the par- 
ticular group of positions being studied. 

In his study of Federal government administrators, Thur- 


10 Cf. Forrest A. Kingsbury. “Psychological Tests for Executives.” Personnel, 
IX (1930), 125-126. 

11 Administrative difficulties have occurred frequently because of changes among 
non-supervisory workers, which resulted in a higher level of mental ability, while the 
supervisors, relatively unchanged, have in many cases been unable to handle these 
workers of superior ability. This situation was noticeable in the early 1930’s when, 
because of economic reasons, a large number of able persons accepted employment in 
low-grade positions. 

12 FE. K. Strong, Jr., in an unpublished study prepared in cooperation with the 
Committee on Public Administration of the Social Science Research Council, found 
that the administrators in technical fields such as law, engineering, medicine, and 
accounting do not generally rate their interests in the top bracket in their own pro- 
fession but have broader interests, mainly in the field of dealing with persons. A 
part of this study has been published: E. K. Strong, Jr. _— of Public Ad- 
ministrators.” Public Personnel Review, VI (1945), 166-17 

13 F. H. Achard and Florence H. Clarke. “ You Can Rise the Probability 
of Success as a Supervisor.” Personnel, XXI (1945), 355. 
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stone found that the Social Scale of the Allport-Vernon Scale 
of Values differentiated among his population better than any 
other measuring device he used.** The Theoretical Scale of 
the Scale of Values also differentiated positively and signifi- 
cantly. The Commercial Interests Scale of Thurstone’s 
Vocational Interest Schedule and the Religious Scale of the 
Allport-Vernon also differentiated significantly but negatively. 

Achard and Clarke, in their study of 300 supervisors in the 
Consolidated Edison Company of New York, prepared a 
special rating scale of the Strong Vocational Interest Blank 
for Men (Revised) and obtained satisfactory results with four 
different types of supervisors. They state that “it was the best 
single all-around indicator of supervisory ability.”* 

Strong has prepared an occupational interest scale for pub- 
lic administrators based on his work with the Forest Service 
and the Committee on Public Administration. The U. S. Civil 
Service Commission, in its test development work in connection 
with the administrative intern program, has obtained promis- 
ing results with the Kuder Preference Record. 

The early studies in this field and the recent investigations 
by Strong, Thurstone, and Achard and Clarke indicate that 
studies of interest, if designed for the particular organizational 
situation, can contribute significantly to administrative and 
supervisory selection programs. In using interest inventories, 
it will probably be found that the appropriate critical scores 
vary significantly among different types of organizations and 
at different levels of the organization’s hierarchy. 

2. Personality Inventories. These inventories can be di- 
vided into two groups; namely, those of the omnibus type 
which furnish several measurements for the individual, such 
as the Bernreuter, and those furnishing a single score, such 
as Laird’s scale on extroversion and introversion. While every- 
one is agreed that, generally speaking, the personality of the 
administrator or supervisor is probably the most significant 
single factor in contributing to successful performance, the 
information now available would seem to indicate that the 


14 Op. cit., pp. 142-145. 
15 Op. cit., p. 362. 
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present personality inventories, either for reasons intrinsic or 
external to them, are only slightly useful for this testing pur- 
pose and are substantially less so than the interest inventories. 
To what extent this result may be produced by the “fudging” 
of responses is not known.*® It would seem that a sophisticated 
group of supervisors and administrators in a competitive situ- 
ation would be able to “beat” these tests. 

Achard and Clarke obtained promising results with the 
Bernreuter, scaled in accordance with their own methods.” 
Their scale differentiated the good from the poor supervisors 
in each of the four groups they were studying better than any 
other test they used except the interest inventory. 

Beckman and Levine obtained favorable results with the 
Allport’s A-S test on a group of supervisory employees of the 
City of Cincinnati.** In a monograph on his work with several 
industrial firms, Hersey reports the use of personality inven- 
tories of the omnibus type for supervisory selection. Kings- 
bury comments in regard to the use of extroversion-intro- 
version tests for this purpose that “since the concepts of 
introversion and extroversion are so ambiguous, and the 
various tests intercorrelate so poorly, further analysis and 
experiment are needed before tests of this type can be accepted 
with confidence for this purpose.”**® Personality inventories 
have frequently been used, with some promising results, for 
comparing leaders with non-leaders in educational institutions. 
Young and Cooper found, for example, in their study of ele- 
mentary school children in the 5th through the 8th grades, that 
the leaders could be characterized as self-sufficient extroverts 
while neither physical nor mental characteristics or interests 
differentiated the leaders from the non-leaders.”° 

Basing a conclusion on the studies made, it is evident that 

16 Jurgensen’s use of paired-comparison and rank-order techniques in his ex- 
perimental form of a personality inventory may contribute to a substantial reduction 
of this factor. See C. E. Jurgensen, “Report on the Classifications Inventory,” 
Journal of Applied Psychology, XXVIII (1944), 445-460. 

17 Op. cit., p. 362. 

18 “Selecting Executives: An Evaluation of Three Tests.” Personnel Journal, 
VIII (1930), 415-420. 

19 Op. cit. p. 127. 


20“Some Factors Associated with Popularity.” Journal of Educational Psy- 
chology, XXXV (1944), 513-535. 








224 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


the present personality inventories can contribute only slightly 
to the selection of administrators and supervisors and that this 
contribution will be further reduced by the fact that candidates 
may “fudge” in a competitive situation. The factors that these 
inventories attempt to measure are so important for successful 
performance that much further work in this area is definitely 
needed. 

3. Mental Abilities. The contribution of this type of test 
to predictions of supervisory success runs the gamut from ex- 
treme importance to only slight or no importance. This result 
is related to the previous discussion where the opinion was 
expressed that one cannot treat these positions as though they 
were identical, and the difference in the results obtained is 
undoubtedly the consequence of differences in job-content plus 
biases among raters in the organizations being studied. 

The results of the Army’s officer selection program indi- 
cate a fairly high positive correlation between the successful 
completion of officer candidate schools and general classifica- 
tion test scores. Thurstone in his study of Federal adminis- 
trators found that the linguistics section of the American Coun- 
cil on Education Psychological Examination made a significant 
and positive differentiation between the better and the poorer 
administrators. Shuman in his study of 99 foremen obtained 
a correlation of +.39+ .07 between the Otis Q. S. Mental 
Ability Beta and performance ratings.** Uhrbrock and Rich- 
ardson, in their thorough study of factory supervisors, found 
mental ability items useful for differentiating between good 
and poor supervisors.** Achard and Clarke found that the Otis 
Self-Administering Test of Mental Ability, Higher Examina- 
tion, was of value for each of the four groups of supervisors 
studied. 

The above brief summary indicates that a mental ability 
test should be included in testing programs for supervisory and 
administrative positions. The weight given to the test should 
differ, however, depending upon the type of positions being 
studied. 


21“The Value of Aptitude Tests for Factory Workers in the Aircraft Engine and 
Propeller Industries.” Journal of Applied Psychology, XX1X (1945), 159. 

22“Trem Analysis: The Basis for Constructing a Test for Forecasting Supervisory 
Ability.” Personnel Journal, XII (1933), 141-154. 
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4. Special Types of Paper-and-Pencil Tests. Mechanical 
aptitude tests have frequently been used, and often with satis- 
factory results, for the selection of factory supervisors. But 
the reason for this successful use has not been fully explained. 
The explanation might lie in the area of the relationship be- 
tween aptitude and interests, with the successful factory super- 
visor being a person whose aptitude and interests are directed 
towards shop and mechanical situations. Whatever the reason 
may be, however, these tests, especially the Bennett Mechan- 
ical Comprehension Test, have proved useful.** 

In preliminary studies of the U. S. Forest Service, the 
Interpretation of Data Test of the Progressive Education As- 
sociation has given some promising results for the selection of 
administrators. In various experimental studies of this test 
on administrative personnel, the candidates have expressed a 
high opinion of its value. 

Uhrbrock and Richardson, in their previously cited study, 
found that test items on the policies and organization of the 
company in which the supervisors were employed made a sig- 
nificant contribution to the selection of supervisors. It would 
be our hypothesis that this type of test has value in selecting 
administrative or supervisory personnel from among tech- 
nicians and mechanics because it would identify those persons 
whose interests lie beyond the technical parts of the positions 
they are occupying. 

A device sometimes used for the selection of supervisors is 
the multiple-choice or true-false form for measuring the ability 
of a candidate to get the right answer to questions on super- 
visory situations. Quentin W. File’s test on How Supervise? 
is one form of this type of test. Hersey has also used a similar 
type of test. It would seem that this type of test measures 
only the logical factor, and not the emotional factor, in human 
behavior. In other words, a supervisor may know exactly how 
to act in a given situation but his emotions may lead to a 
different procedure. This statement is not meant to deny that 
this type of test has value but that its primary purpose should 


23 See the previously cited studies of Shuman and Achard and Clarke. 
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probably be to determine the need for training. Its value for 
selection purposes has not as yet been determined. 

Thurstone in his study of Federal administrators obtained 
significant differences between the good and poor groups on 
three special types of paper-and-pencil tests, Estimating, Clas- 
sification, and Gottschaldt figures. The Estimating Test re- 
quires the candidate, by using logical methods rather than 
knowledge, to arrive at a statistic. For example, a candidate 
might be asked the number of telephones in use for non- 
commercial purposes in the United States for the year 1941. 
It would be highly improbable that any candidate would know 
the answer to this question. He would have to rely on esti- 
mating to arrive at his answer. In the Classification Test, the 
candidate is asked to sort a number of cards, each containing 
the name of a prominent individual, into as many groups as 
he wishes. Thurstone found that the better administrator 
used fewer categories. The Gottschaldt Figures Test is in 
multiple-choice form and requires the candidate to identify in 
which one of five large, complicated diagrams there exists a 
smaller figure. The results obtained by Thurstone justify 
further study of the value of these tests for selecting adminis- 
trative and supervisory personnel. 


Oral Interviews 


At some stage in the selection of supervisory and adminis- 
trative personnel, an interview is practically always used. But 
there is no evidence available as to its value. The Army Ser- 
vice Forces is now engaged in a program which indicates, in at 
least tentative form, that by refining the questions and rating 
forms used, moderate validity is obtained for this testing 
device. 

The British Army has been using what might be called a 
group interview in which the candidates are observed in their 
discussion of a subject selected either by or for them. From 
observation of this type of interview it would seem that this 
method, when properly administered, offers valuable informa- 
tion for administrative and supervisory positions. 

One of the most common difficulties with the administration 
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of an oral interview is the need for careful selection and train- 
ing of raters. Quite often the raters are selected on a chance 
basis rather than because of their technical competency. Too 
frequently, also, the training of the raters before the interview 
has been inadequate in terms of the complexity of the task 
assigned to them. 


Ratings in Training Courses 


In those organizations which conduct training programs 
for administrative and supervisory personnel, objective ratings 
on performance in these programs should furnish valuable addi- 
tional information for selection for promotion. The standard 
conditions under which these training programs can be given 
would be a definite factor in helping to make these ratings 
valuable for this purpose. In the promotion examination pro- 
gram adopted recently for shop supervisory positions in Navy 
field establishments, a rating in the work improvement program 
is given a weight in determining rank on the eligible register. 
This examining program is under the direction of the U. S. 
Civil Service Commission, and includes, in addition to the work 
improvement program rating, a written test on administration, 
an oral interview, an evaluation of experience and a perform- 
ance rating. 


Evaluation of Biographical Data 


Public personnel agencies use extensively the evaluation of 
a candidate’s background, in terms of experience and training, 
as a method for rating applicants for administrative and super- 
visory positions. To the best of our knowledge, there is no 
information available as to the validity of this test for this 
type of position. If this testing method is based on much more 
extensive information than is usually contained on an applica- 
tion blank, it should provide a sound basis for selection if, in 
addition, improvements can be devised in the method for trans- 
lating this information into ratings. Intensive study of the 
validity of this testing method is needed. 

Basic to the success of any study of the value of tests is the 
preparation of a valid criterion. This problem is especially 
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complex in evaluating administrative and supervisory posi- 
tions. Practically speaking, in devising tests for one organiza- 
tion, an attempt at agreement within the organization should 
be made as to the standards to be used in rating. This recog- 
nizes that different standards in use by another organization 
will invalidate the results obtained in the first instance. It can 
be pointed out that in the most successful studies in this field 
as much time has been spent in obtaining good ratings as on 
the testing program itself. 


Conclusions 


The conclusions we would draw from this summary of 
trends in testing for administrative and supervisory posi- 
tions are: 

(1) The testing program should include a mental ability 
test and an interest inventory. 

(2) Further study is needed of the value of personality 
inventories, tests of company policies and organization, and 
special types of tests such as the /nterpretation of Data and 
the Thurstone Estimating Test. 

(3) The value of ratings in training courses and on bio- 
graphical data should be explored. 

(4) Precise analysis of the jobs being studied is an absolute 
need in order to identify homogeneous sub-groups. 

(5) Differences in the value of particular tests should be 
expected for different supervisory and administrative jobs. 

(6) The evident practical urge to include oral interviews 
makes necessary further improvements in this testing tech- 
nique. 









































ARMY GENERAL CLASSIFICATION TEST SCORES 
FOR CIVILIAN OCCUPATIONS 


THOMAS W. HARRELL? 


University of Illinois 
and 


MARGARET S. HARRELL 


For more than twenty years research workers have been 
referring to data on intelligence and occupation based on the 
Army Alpha scores and the civilian occupations of enlisted men 
in World War I.? The present study deals with similar mate- 
rial. It reports the Army General Classification Test (GCT) 
scores of 18,782 white enlisted men of the Army Air Forces Air 
Service Command® distributed according to their previous 
civilian occupation. Means and medians and standard devia- 
tions are reported for the 74 occupations for which there were 
samples of sufficient size to be of significance. 

The data, more applicable to occupations of today than are 
the data presented in earlier studies, supplement present 
knowledge concerning ability levels for occupational groups 
and are of value in educational and vocational counseling of 
civilians and especially, at this time, of discharged soldiers. 

There has been much discussion as to how well the Army 
Alpha sample represented the general population. At the 
present time it is impossible to decide that issue with respect 
to this sample. It is possible that the averages among the 
professional occupations are too low since conceivably many 

10On leave, Major, U. S. Army Air Corps, Director, Manning Section, Head- 
quarters 15th Air Force. 

2 Yerkes, R. M. “Psychological Examining in the U. S. Army.” Memoirs of 
the National Academy of Sciences, 1921. 

Fryer, Douglas. “Occupational-Intelligence Standards.” School and Society, 
XVI (1922), 273-277. 

Bingham, Walter V. Aptitudes and Aptitude Testing. New York: Harper 
Bros., 1937, pp. 44-59. 

3 Data were furnished by Lt. Col. R. W. Faubion and Lt. Col. J. L. Webster. 
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of the best men in the profession would have been officer mate- 
rial. It is more likely that the averages among the lowest 
scoring occupations are too high since for a while the Army 
Air Forces were receiving a smaller percentage of enlisted men 
with low scores on the GCT than were the Army Ground and 
the Army Service Forces. 

Scores were obtained from Informational Rosters made up 
from Soldier’s Qualification Cards and other sources. Only 
those cases were included where the roster clearly indicated the 
previous civilian occupation of the soldier by job title. The 
job titles in the following tables are those given and described 
in Army Regulation 615-26, Index and Specifications for 
Civilian and Military Occupational Specialists, or in the U. S. 
Employment Service Dictionary of Occupational Titles. In a 
few instances, however, a general job title has been used in 
this study instead of the more specific breakdowns of the job 
described in A.R. 615-26, i.e., Engineer includes mechanical, 
civil and mining engineers. These general job titles are Engi- 
neer, Laboratory Assistant, Inspector, Musician, Foreman, 
Electrician, Assembler and Welder, Manager, and Miscellane- 
ous. Miscellaneous is a general title which includes managers 
of various business establishments, i.e., moving picture theaters, 
bowling alleys, etc., but excludes managers of retail stores. 
Manager, Retail Store includes managers of both chain and 
independent retail stores. Mechanic includes all kinds of me- 
chanics except air-plane and automobile mechanics. 

The desired maximum size of sample for a single occupation 
was 500 cases, and when that maximum was reached further 
tabulation of the occupation was discontinued except in a few 
instances where the range of scores was great. For the major- 
ity of the occupations 500 cases were not available. It was 
originally planned‘to omit all occupations with less than 100 
cases; some have been included, however, as they involve 
several professional groups in which there is considerable 
interest. 

In order to provide a rough check on the reliability of the 
size of the sample, the scores for each occupation were tabu- 
lated in two distributions, A and B, of approximately the same 
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OCCUPATIONS 
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Mean and Median GCT Standard Scores, Standard Deviations and Range of 


Scores of 18,782 AAF White Enlisted Men by Civilian Occupation 











Occupation N M Median caeens Range 
I os neko iccdkeceehenea tani 172 128.1 128.1 11.7 94-157 
EE Dadshcaiws accep easerenioned 94 127.6 126.8 10.9 96-157 
I i ica GS. sca ial draw alae s 39 126.6 125.8 11.7 100-151 
Public Relations Man .............. 42 126.0 125.5 11.4 100-149 
RES ES iy errr 62 125.9 125.5 11.2 98-151 
A bd. cue, 5 catia kes-dklbn 6onkeet 21 124.8 124.5 13.8 102-153 
as icc s am ka-aco kale bas wie 45 124.5 125.7 11.7 100-157 
nik an dee Hawa. oc 8,0ae 165 124.2 124.5 11.7 88-153 
EE So re ere re 256 122.8 123.7 12.8 76-155 
Draftsman ..... pampbat ava awemsees 153 122.0 121.7 12.8 74-155 
I n.d se kewisbsensnasa kas 147 121.0 121.4 BS 66-151 
A errr reer 58 120.5 124.0 15.2 76-149 
Tabulating Machine Operator ....... 140 120.1 119.8 13.3 80-151 
a ai Linke ela dwede esi 272 120.0 119.7 13.1 70-157 
I, ccs cup coaddewdeaes 42 119.0 120.7 11.5 90-137 
Parchasme Agent ....2..ccccccccess 98 118.7 119.2 12.9 82-153 
Manager, Production .............. 34 118.1 117.0 16.0 82-153 
oe ads ho: wccnnisid's anak 95 117.6 119.8 13.9 66-147 
SE EE, f vciccvcsessebanreoe 496 117.5 117.9 13.0 68-155 
ID dirs 5G bavidac sven Saucces 468 116.8 117.3 12.0 80-147 
Manager, Miscellaneous ............. 235 116.0 117.5 14.8 60-151 
Installer-Repairman, Tel & Tel ...... 96 115.8 116.8 13.1 76-149 
EN i ea Ghee Ru aad aa pa'a adie beh 111 115.8 116.8 11.9 80-145 
Instrument Repairman .............. 47 115.5 115.8 11.9 82-141 
SUED MGUOUOEEM oiscdsecciesesnevee 267 115.3 116.5 14.5 56-151 
Printer, Job Pressman, Lithographic 
DY ccicccrecseueeaneeeasaes 32 115.1 116.7 14.3 60-149 
NN ines oe sms oh as wimewade 494 115.1 116.2 15.7 60-153 
Ceca ke emia adeekbheks es 48 114.9 115.4 11.2 82-139 
Manager, Retail Store .............. 420 114.0 116.2 15.7 52-151 
Baboeatory ASStONt .......csscccecss 128 113.4 114.0 14.6 76-147 
NS PORT PERT er ee oe 60 112.5 111.6 12.5 76-143 
 Giig 4 Shaicis Newb as bead mae oa 358 112.3 113.1 15.7 54-147 
ES eer eee rere 490 111.8 113.0 16.3 54-151 
Receiving and Shipping Clerk ........ 486 111.3 113.4 16.4 58-155 
CE (ccnancaveceenceetncss waved 157 110.9 112.8 15.9 56-147 
DN ccciccksaweceue Gactideeas 456 110.1 110.8 16.1 38-153 
EER A ee oe ee eae 298 109.8 111.4 16.7 60-151 
en tc avnndcckhewe sa 56 109.8 113.0 14.7 68-147 
MENS BEOCRAMIE oc ccc ccccsccsens 235 109.3 110.5 14.9 66-147 
ee rd a als nh oi didiaeeiss 492 109.2 110.4 16.3 42-149 
Riss iG d a ckeead s waae oon 289 109.0 110.6 15.2 64-149 
ESOC LECT T EE 172 108.5 109.4 15.5 64-147 
Receiving & Shipping Checker ........ 281 107.6 108.9 15.8 52-151 
Sheet Metal Worker ................ 498 107.5 108.1 15.3 62-153 
107.1 108.8 15.5 


Lineman, Power and Tel & Tel ...... 77 


70-133 











232 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


TABLE 1 (Continued) 











‘ . Standard 

Occupation N M Median dasiiion Range 
ea an Se wade weadenees 498 106.3 106.6 14.6 48-145 
SR ire ee iene 421 106.3 108.3 16.0 60-155 
IN oo viivcacncnenan sxe 486 104.8 105.7 17.1 42-151 
PED. cc cctnncases}escuns 539 104.2 105.9 16.7 30-141 
sn Sa Se Peace prem 239 104.1 105.3 15.1 50-141 
OPEC OEE LOCI EOE 48 103.5 104.7 15.9 66-127 
Usted, ciscendadmncesonnes 59 103.3 105.8 14.5 68-131 
4 tea aoa Sig Sin a ma Si algsaiale wide 259 102.9 104.8 17.1 42-147 
as ce cs ok ae pala aches doe 128 102.7 104.8 16.0 56-139 
ESTEE a Se ee eee eee 98 102.2 105.0 16.6 56-137 
Carpenter, Construction ............ 451 102.1 104.1 19.5 «4-147 
NE i odode bndenccncacsacwens 72 101.9 105.2 18.0 56-139 
EEE ES eS ae a 493 101.8 103.6 16.1 48-147 
SE US ccdcwacccecsssecses 466 101.3 101.8 17.0 40-151 
NL PSM ASR 79 101.1 105.5 20.2 48-137 
A ene eee nee 194 100.8 103.0 18.4 46-143 
NE, acca geass saiues® 354 99.5 101.6 19.1 42-147 
RE os ccc cicadsdecducess 440 98.3 100.1 18.7 38-147 
Crane Hoist Operator .............. 99 97.9 99.1 16.6 58-147 
MN cs tcnsacdekaadons 436 97.2 99.5 20.8 20-147 
ae a ik a 56 97.0 97.3 17.7 50-135 
EE deci henssnnatanusies 817 96.2 97.8 19.7 16-149 
EG i eces cs cagawadekinensed 856 95.8 97.7 20.1 26-145 
NS ee ee Re i ee ce hs 103 95.3 98.1 20.5 42-141 
PE ocbsicitsaedevinsideawae 59 94.7 96.5 19.8 46-137 
NN IG th Sheds canny dist anlgua ed 700 92.7 93.4 21.8 24-147 
id 6 es alae ad epee ined 817 91.4 94.0 20.7 24-141 
DC cis hecsnkieuessenke bane ds 156 90.6 92.0 20.1 42-139 
SR oe ne meee eee 77 87.7 89.0 19.6 46-145 





size. The difference between the m.1ns of A and B for each of 
the 48 occupations with 100 or morc cases ranged from 0.1 to 
5.1 and averaged 1.7. (The median lifference was 1.3.) For 
the 26 occupations with less than 100 cases, the difference be- 
tween the means of groups A and B ranged from 0.4 to 11.7 
and averaged 3.7. (The median difference was 3.0.) 

Table 1 gives the mean, median, standard deviation and 
range for 74 occupations. The means and medians were cal- 
culated from distributions grouping scores by intervals of two. 
The standard deviations were calculated from distributions 
grouped to provide from 12 to 18 class intervals. 

Table 2 gives the percentage distribution of each occupa- 
tion. It shows, as have other similar studies, the great over- 
lapping of scores between occupations. Even Teamster, the 
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TABLE 2 
Percentage Distribution of GCT Standard Scores by Civilian Occupation of 18,782 AAF White Enlisted Men 
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5 130-139 
100-109 
2 60- 69 
50— 59 
40- 49 
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20- 29 
10- 19 
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Engineer 


1 
1 


Pub. Rel. Man 
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Tab. Mach. Op. 


Bookkeeper 


Sales Mer. 
In.-Rep., T & T 


Photogrphr. 
Cashier 


Pharmacst 
Purch. Agnt. 
Prod. Mer. 
Clk., Gen’l 
Clk.-Typist 
Mgr. Misc. 
Inst. Rep. 


Steno. 





| 
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Printer 


Tool Maker 


Mer. Ret. Str. 
Inspector 


Radio Rep. 
Saleman 
Artist 

Lab. Asst. 


nN 


i» 


to] 


Stock Clerk 
R/S Clerk 


Musician 


' 


| 


Machinist 
Foreman 


| CON | COMM | 


Watchmaker 


{ 


* Where the critical ratio is 3 or more, no entry has been made in the table. 
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TABLE 3 (Continued) 


' Critical Ratios between the Means of 74 Civilian Occupations (N=18,782 White 
ite Enlisted Men, AAF) 








Airpl. Mech. 
Sales Clerk 

Sht. Met. Worker 
Lineman 


Electrician 
Lathe Operator 
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R/S Checker 
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Mechanic 
Machine Oper. 
Auto Service. 
Riveter 
Cabinetmaker 
Upholsterer 
Butcher 
Plumber 
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Machinist 
Foreman 
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Tool Maker 
Inspector 
| Stock Clk. 
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Musician 
Machinist 
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Airpl. Mech. 
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* Where the critical ratio is 3 or more, no entry has been made in the table. 
** The following are additional critical ratios for Watchmaker: Watchmaker and Prod. 
725 Mgr., 2.5; and Mgr., Misc., 2.8; Tel & Tel Installer-Repairman, 2.5; Cashier, 2.6; Instrument 
Repair., 2.2; Radio Repair., 2.6; Printer, 2.3; Salesman, 2.5; Artist, 2.0; Mgr. Retail 
Store, 2.0. 
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occupation with the lowest mean, has 2% of its cases scoring 
as high as the mean of Accountant, the occupation with the 
highest mean. On the other hand the top seven occupations 
have no cases as low as the mean score for Teamster. Evi- 
dently a certain minimum of intelligence is required for any 
one of many occupations and a man must have that much 
intelligence in order to function in that occupation, but a man 
may have high intelligence and be found in a lowly occupation 
because he lacks other qualifications than intelligence. 

Table 3 gives the critical ratios between the means of 
each occupation compared with every other occupation. 
Where the critical ratio was 3 or more no entry has been made 
in the table. The table shows, as does Table 2, both the over- 
lapping and differentiation between occupations. In general, 
the mean of an occupation is not significant within a range of 
approximately + 10 occupations where the occupations have 
been arranged in descending order of means. This range 
would have been less if all of the occupations had had at least 
100 cases. It is all the more interesting, consequently, that 
Chemist with 21 cases, Engineer with 39, Public Relations Man 
with 42 and Reporter with 45 cases have means significantly 
higher than those of at least 49 other occupations. 

As is always found in studies of this kind, the professional 
and clerical administrative groups of occupations show the 
highest average scores. Those occupations with average scores 
of 120 or more are, in the order of their superiority: Account- 
ant, Lawyer, Engineer, Public Relations Man, Auditor, Chem- 
ist, Reporter, Chief Clerk, Teacher, Draftsman, Stenographer, 
Pharmacist, Tabulating Machine Operator and Bookkeeper. 

Since the GCT is a measure of ability to manipulate words, 
numbers and space relations, it is to be expected that those 
occupations with the lowest averages on the test are likewise 
the occupations least concerned with words, numbers or space 
relations. Lowest scoring occupations from low to high are 
Teamster, Miner, Farmhand, Farmer, Lumberjack, Barber, 


Laborer and Truck Driver. 











MECHANICAL ABILITY, ITS NATURE AND MEA- 
SUREMENT. I. AN ANALYSIS OF THE VARI- 
ABLES EMPLOYED IN THE PRELIMINARY 
MINNESOTA EXPERIMENT 


J. R. WITTENBORN 
Yale University 


A Review of the Minnesota Program 


THE present study comprises a brief review of the prelim- 
inary Minnesota experiment and a factorial analysis of the 
variables. The purpose of this paper is to apply modern an- 
alytical techniques to the interesting problem of the nature 
of mechanical ability which was begun some two decades ago 
by the authors of the Minnesota experiment (5). 

Although the Minnesota research had as its primary aim 
the discovery and measurement of mechanical ability, the 
term “mechanical ability” has never been rigidly and unam- 
biguously defined. Since what is meant by mechanical ability 
has never been perfectly general, procedures for assaying the 
degree of mechanical ability are held to be appropriate by 
various investigators to varying degrees. In general, however, 
mechanical ability has been an expression for whatever ability 
or abilities are required for creditable work with tools and 
machinery, i.e., mechanical work. The exact nature of me- 
chanical ability was not specified by the authors of the 
Minnesota study; it was instead likened by them to intelli- 
gence. It was considered to be a general function, probably 
comprising constituent parts, the precise identity and impor- 
tance of each varying from authority to authority. 

The question of the organization and composition of me- 
chanical ability is, of course, critical for economical testing 
procedures. Since the publication of the Minnesota studies, 
new analytical techniques have been devised and employed. 
241 
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It is possible that by combining the contributions of various 
investigators a tentative answer to the question of the organi- 
zation of mechanical ability may be offered. The answer 
must be regarded as tentative not only because our researches 
to date are by no means exhaustive but also for a more positive 
reason. The performances which investigators have found 
desirable to scrutinize and to measure are determined by the 
requirements and the characteristics of our culture. As 
mechanical and other practical operations change perhaps old 
requirements become less important and new ones rise to 
importance. Different technical devices will be developed and 
the precise composition of that loosely defined group of abilities 
referred to as “mechanical” must be expected to change. 

Of the several patterns of speculation concerning the com- 
position of abilities, the theory of unique traits has been most 
provocative. This theory has been linked with such names as 
Hull, Kelley, Thurstone, Thorndike, and Woodrow. It, in 
general, provides two assumptions: (1) that all variability in 
human behavior may be expressed as a function of a limited 
number of independent, elemental abilities and (2) that these 
abilities may be discovered and suitable tests designed. ‘This 
theory had been influencing psychometric research some time 
prior to the publication of the Minnesota Mechanical Ability 
Tests. Its full effect was not felt, however, until the invention 
of a practicable factor analysis by Thurstone. Before the 
development of effective factorial methods, a variety of ex- 
pedients were employed by those who desired to make explora- 
tions in the area of human ability. 

In the preliminary experiment, the literature was carefully 
surveyed for the purpose of selecting tests which were known 
to possess encouraging validity and reliability. The practical 
problems involved in their administration were considered. 
Particular effort was made in the selection to avoid including 
more than one test which appeared to measure the same aspect 
of mechanical ability. Twenty-six tests were selected, reor- 
ganized, and in some cases greatly changed with respect to 
instructions, conditions of administration, or length. Although 
the authors claimed that test individuality was emphasized in 


— 


ree 





DUS 
ni- 
ver 
nes 
ive 


Id 


eee ma. S eS 


~ = © 





remem Eg TT 


a 





MECHANICAL ABILITY, ITS NATURE AND MEASUREMENT 243 


the selection, it was possible for them to make a provisional 
classification of the tests under seven general headings. The 
headings indicated the nature of the operation which the tests 
appear to be measuring (5, 43-44). 


I. Standard group intelligence tests 


1. 
p 


Army Alpha, Form 6 (group paper) 
Otis Self-Administering Tests of Mental Ability, 
Higher Examination: Form A (group paper) 


II. Simple motor tests 


1. 
. Tapping Test B (group paper) 


Wn WHO 


A 
8. 


Tapping Test A (group paper) 


Tapping Test C (individual apparatus) 

Steadiness of Motor Control (individual apparatus) 
Accuracy of Movement or Tracing Paper (group 
paper) 

Accuracy of Movement or Tracing Board (individual 
apparatus) 

Aiming (individual paper) 

Speed of Movement (group paper) 


III. Balancing tests 


3 
> 


Body Balancing (individual apparatus) 
Stick Balancing (individual apparatus) 


IV. Complex eye-hand coordination tests 


1. 
2. 
3. 
4. 


Link’s Machine Operator’s (individual apparatus) 
Card Sorting (individual apparatus) 

Card Assembly (individual apparatus) 

Packing Blocks (individual apparatus) 


V. Assembly tests involving manipulation and responses to 
spatial relations 


1. 
2. 
3. 


Stenquist Assembly (group apparatus) 
Paper Form Board (group paper) 
Link’s Spatial Relations (individual apparatus) 


4. Cube Construction (group apparatus) 


VI. Tests of mechanical knowledge 


1. 


Stenquist Picture Tests I and II (group paper) 
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VII. Miscellaneous tests 
1. Slow Movement or Motor Inhibition (individual 
paper) 
Digit-Symbol Substitution (group paper) 
Letter Cancellation (group paper) 
Number Cancellation (group paper) 
Rhythm or Perception of Time (group apparatus) 


WW bh 


It will be of interest to compare the authors’ tentative 
classification of the tests with the functional classification dis- 
covered by the present writer’s analysis. 

The tests were administered to boys who were enrolled in 
the seventh and eighth grades of a Minneapolis junior high 
school and whose curriculum included the shop courses. A 
number of factors determined the selection of this particular 
group of subjects. Boys were chosen rather than men because 
it was felt that in boys individual differences would be de- 
termined less by differences in the amount of mechanical 
training and more by stable abilities which accrue from gen- 
eral sources. Boys from the middle class were selected rather 
than a more heterogeneous sample. The reason for this 
selection was that in all probability the boys in the upper 
classes have had a restricted opportunity to acquire mechan- 
ical abilities and the boys from the low, under-privileged class 
have had restrictions in their development, also. Selection of 
boys who were enrolled in shop courses was determined by the 
need for a suitable criterion which could not only be reliably 
determined but which would also suitably represent the varied 
operations which are considered to comprise mechanical 
ability. The battery of 26 tests required 11 hours of testing, 
and complete data were collected for 217 boys. 


The Original Analysis of the Variables of the Preliminary 
Experiment 


The authors of the Minnesota study were interested in the 
organization of mechanical ability because of its implication 
for this very practical question: Is it possible for an individual 
to be distinctly gifted in one line of mechanical work and to 
possess poor or mediocre capacity in others, i.e., is there a 
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mechanical ability or group of abilities? In their attempt to 
answer this question, they intercorrelated all of the variables 
employed in the preliminary experiment and then subjected 
these intercorrelations to analytical procedures. Since Spear- 
man’s theory of general ability, particularly with respect to 
intellectual functions, was one of the most arresting theories 
of the day, the authors concerned themselves principally with 
the task of finding evidence of a general mechanical ability. 

In order to fulfill Spearman’s demand that only dissimilar 
tests be subjected to scrutiny when a general factor is sought, 
the authors carefully examined their own tests and selected a 
group of 15 tests which were dissimilar to each other and were 
not correlated with each other to an exceptional degree. In 
light of current factor theory, the implications of this selective 
procedure are obvious. Similar tests are likely to be those 
highly intercorrelated and if several highly intercorrelated 
tests are encountered in a group of tests, we are certain to find 
a cluster or a factor. If such a cluster were found, it would 
not be possible to account for all of the inter-test correlations 
in terms of one general factor. 

The intercorrelations of the selected 15 variables were ar- 
ranged in the usual tabular form, and the inter-columnar cor- 
relations for this table varied from minus .68 to plus .81. It 
is obviously impossible to find any hierarchical order in such 
a table. A similar lack of hierarchical arrangement was en- 
countered when the variables were corrected for attenuation. 
The results of this procedure combined with the results of 
certain preliminary procedures forced the authors to the con- 
clusion that their data offered no evidence for a general factor 
in mechanical ability. 

The authors then sought evidence for the existence of group 
factors. On a trial-and-error basis, attempts were made to 
construct hierarchies by arbitrarily selecting certain tests for 
grouping. This procedure was adopted because group factors 
are general within their own restricted field, that is, variables 
which define a group factor may be arranged in hierarchical 
order. The authors assumed that if several well-defined hier- 
archies were disturbed when tests from any other hierarchy 
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were included, the evidence would be strongly in favor of the 
existence of group factors, that is of mechanical abilities rather 
than a general mechanical ability. The writers reported that 
they found seven perfect hierarchies of four tests each. Two 
of these hierarchies were found to be mutually exclusive with 
respect to their component tests. The other five hierarchies 
drew upon the same tests. The writers do not indicate what 
variables contribute to all of the hierarchies. Since such hier- 
archies can be arbitrarily constructed without one’s knowing 
to what degree they overlap, the analysis of the organization 
of mechanical ability provided by the Minnesota study leaves 
much to be desired. From their presentation it may be in- 
ferred that if a general mechanical ability factor is present, it 
is certainly not sufficient to account for all of the inter-test 
correlations. It may be further inferred that group factors 
do exist, but the degree of independence which they possess 
and their exact nature is in no way manifest. 


A Factor Analysis of the Variables in the Preliminary 
Experiment'* 


The Minnesota investigators had employed numerous vari- 
ables which are representative of if not identical with many in 
use today. Their population was quite large and in their 
attempts at the analysis of the organization of mechanical 
ability they determined the intercorrelations among all of the 
variables. An analysis of the Minnesota data using devices 
which were not then available is in order in two respects: (1) 
to round out the classic Minnesota investigation and (2) to 
shed light on the current problems of measurement of mechan- 
ical ability. As a consequence the intercorrelations of the 
Minnesota variables given in Table 1 were submitted to a 
centroid analysis; seven factors were extracted, six of which 
are given in Table 2. The centroid analysis was made for 
twenty-seven variables. The tests were the original list of 
twenty-six variables with the exception of the cube construc- 
tion test, which was not intercorrelated by the Minnesota 


1 This analysis is concerned only with the variables employed in the preliminary 
experiment. In a forthcoming publication additional analyses of the Minnesota 
data will be reported. 
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authors with the other variables. This left twenty-five tests. 
The Stenquist Picture Test, however, really comprised two 
parts, I and II. Each was scored separately, correlated with 
the other variables separately, and the two parts are treated 
as two variables in the present study. Age was also included 
as a variable and thus the total list of twenty-seven variables 
which the present writer has analyzed is comprised. 


TABLE 2 
Centroid Matrix 








A B Cc D E F h2 








1 00 25 -.19 — .37 -.2 — .08 32 
2 29 .06 26 .14 03 - .20 21 
3 34 — 46 35 25 43 25 76 
4 31 .09 23 .26 -.22 —.09 28 
5 50 18 — .24 31 -.10 13 46 
6 53 12 -.22 49 -.10 20 63 
7 48 .26 - .09 .22 22 09 41 
8 44 .16 — 35 -.10 48 - .25 64 
9 50 27 — .06 17 —.15 18 41 
10 52 - 30 — .28 .08 — .02 07 45 
11 40 28 — .38 — .13 33 - .37 64 
12 35 — 46 31 18 29 27 62 
13 49 — .05 -.20 18 10 15 35 
14 48 —.47 -.17 — .04 -.11 08 50 
15 14 06 Ri] .20 10 — .09 10 
16 23 -.11 —.22 — .05 10 — .08 13 
17 44 29 .10 -.27 03 11 37 
18 27 08 .23 .08 — .04 — .26 22 
19 43 — .36 -.25 — .28 —.18 — .04 48 
20 30 — .42 -.18 -.29 - .20 -.15 44 
21 57 56 - 15 —.22 — .06 05 71 
22 20 29 15 13 -.21 — .03 20 
23 33 32 17 — .32 14 16 37 
24 34 — .38 .08 — .44 — .16 .34 55 
25 A4 .29 R 2 — .35 —.18 .23 50 
26 30 -.11 .06 17 -.27 — .32 30 


27 19 — .04 23 .05 -.11 -.27 17 





The centroid factors were orthogonally rotated according 
to the well-known principles of maximizing the number of 
zero loadings and minimizing the number of negative ones. 
After thirty-six sets of rotations had been made it was found 
that further rotations would contribute little to the simplicity 
of the patterns revealed. Each of the six orthogonal factors 
is well defined, and for the first five factors the loadings are 
exceptionally high. The rotated factor matrix is given in 
Table 3 and the transformation matrix is given in Table 4. 
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TABLE 4 


Transformation Matrix 











A B Cc D E F 
I IS — .64 - 41 — 45 - 31 04 
II 45 49 39 — .54 —.20 .24 
III 35 — 48 51 24 Al 39 
IV 62 25 — .35 61 —.24 .08 
V Be 1S -.27 — .23 74 — 42 
VI -.21 17 - .47 -.11 .29 78 





In order to clarify the implication of these factors, each will be 
discussed with respect to its component variables and with 
respect to the degree to which component variables are satu- 
rated with other factors. 

Since the factors are discussed variable by variable, the 
data will be presented in the form of factorial equations.” 


Factor I—Spatial Visualization 
I II III IV V VI U2 


10. Link’s Spatial Relation ...... 21 .00 05 .16 .02 01 55 
14. Paper Form Board .......... 35 .00 .08 .06 .00 .00 51 
19. Stenquist Assembly ........ 44 1 .00 02 01 01 51 
20. Stenquist Picture I ........ 40 .00 .00 .00 Ol 04 55 
21. Stenquist Picture II ........ Jo .00 Jl .02 02 01 .29 


The Spatial Relations Test of Link is the original form of 
the currently popular Minnesota Spatial Relations Test. The 
Paper Form Board Test from the Army Alpha is now modified 
and known as the Minnesota Paper Form Board Test. The 
Stenquist Assembly Test has been lengthened and modified 
and appears as the Minnesota Mechanical Assembly Test. 
Twenty-one per cent of the total variance of the Spatial Rela- 
tions Test is due to the spatial factor. Another sixteen per 
cent of its variance is due to factor IV, which is identified as 
a dexterity factor. Users of the Minnesota Spatial Relations 
Test have long believed that this test calls for spatial ability 
and for a certain amount of dexterity or manipulative ability 
as well. The Paper Form Board Test has thirty-five per cent 
of its variance attributable to the Spatial factor; about eight 


2 Factorial equations are more revealing than simple factor loadings because 
they not only show how much of the total variance of a test is due to each factor, 
but they also indicate how much of the variance is not due to common factors, 
ie., how much is unique (U?) to the test in the present test sample: the values 
in the factorial equations are equal to the respective factor loadings squared. 
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per cent is due to factor III, which in this study is called the 
scholastic ability factor. The Assembly Test is largely a 
measure of spatial ability. It is very interesting to observe 
that it calls for the dexterity factor, IV, to a negligible degree. 
Contrary to the implications of its name, the Assembly Test 
calls for the ability to visualize the relations of parts rather 
than the ability to excel in the manual process of assembling 
parts. 

Some explanation should be offered for the difference 
between the saturations of tests 20 and 21, both of which are 
Stenquist picture tests and contribute greatly to the spatial 
factor. It is observed that test 21 owes approximately 10 per 
cent of its total variance to the scholastic ability factor. A 
possible explanation of this difference between the two picture 
assembly tests is that test 21 was given under marked restric- 
tion of time. The time for each exercise in the test is reduced 
twenty per cent. The meaning of time restriction for this 
saturation of test 21 is a question that will be discussed further 
under factor III. 

Factor II is determined by tests which appear to call for 
a high degree of speed in certain stereotyped ballistic move- 
ments of the wrist and forearm. 


Factor I[I1—Stereotyped Movement 


I II Ill IV V VI U2 
(9. Link’s Mach. Operator) .... .00 .10 01 30 .00 .00 59 
17. Speed of Movement ........ .00 30 .00 02 04 01 63 
ee ON Rarer rer .00 31 01 .00 05 Ol 62 
i OME ccc ccdiddeeeaees 01 53 .00 01 .00 .02 43 
QE ES ndcciocienedence .02 48 .00 .02 .00 .00 48 


The Speed of Movement Test calls for as many vertical 
marks as possible within a time limit. Variables 23, 24, and 
25 were all tapping tests which were conducted in different 
ways. In test 23, the individual is to make one dot in each 
of the large number of tiny squares. In test 24 he is to make 
as many dots as possible in each of the large squares outlined 
on his test paper, fifteen seconds allowed per square. In test 
25, the individual is equipped with a metal plate and a stylus 
and his rate of tapping is determined. In all four of these 
tests the individual must make very rapid movements of the 
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wrist and the forearm. It is observed that test 9 had ten 
per cent of its total variance attributable to this factor. The 
machine operator’s test, 9, involves dropping balls into a 
funnel at intervals which permit the ball to drop through a 
momentarily appearing aperture in a platform beneath the 
funnel. This performance calls for a vertical dropping move- 
ment of the hand which is not unlike the vertical movements 
called for in the various tapping tests. 

Factor III is called the Scholastic Ability Factor for the 
simple reason that two well-known tests of scholastic ability 
comprise it. 

Factor I1I—Scholastic Ability 


I II Ill IV V VI U2 

RS OS Ee ee eee rere .03 08 -.20 .00 .00 .04 65 
‘3 aoaee rer rere ee .00 01 J3 .00 01 .00 25 
I hres ahd were donee 02 .00 59 .00 .00 .00 39 
(21. Stenquist Picture II) ........ 55 .00 All 02 .02 01 29 


These tests are the Army Alpha and the Otis. The scores 
are not intelligence ratios but are simply raw scores. As 
would be expected, this factor is negatively correlated with age. 
It will be remembered that the subjects for the preliminary 
experiment were picked from the seventh- and eighth-grade 
classes. Ordinarily the older children in a pyblic school class 
are the children who have been retained, and most frequently 
this retention has been due to dullness on the part of the child. 
If this general observation may be validly applied here 
accounts for the negative relation between this factor and age. 
It is further observed that variable 21, the Stenquist Picture 
Test II, has a probably significant relationship with this factor. 
The next variable most highly correlated with this factor is 
variable 14, the Paper Form Board Test. The Army Alpha, 
the Otis, the Paper Form Board, and the Stenquist Picture 
Test II are all pencil-and-paper tests which may be classified 
as mental tests and all four are highly speeded. Perhaps these 
common characteristics can account in part for the positive 
intercorrelation. 

Factor IV is determined by tests which appear to call for 
a type of manual dexterity. 
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Factor [V—Manual Dexterity 
I II Ill IV V VI U2 


(4. Body Balancing) ........... 01 .02 02 mo .02 01 Re 
5. Card Assembly, Time ....... .00 01 .00 43 01 01 54 
6. Card Sorting, Time ........ .00 .00 03 60 .00 01 36 
NS ara Tre .02 03 .03 .23 10 01 58 
9. Link’s Mach. Operator ...... .00 .10 01 30 .00 .00 59 
10. Link’s Spatial Relation ..... 21 .00 05 .16 02 01 B. 
ee .03 .00 .06 21 .04 01 65 


Variables 5 and 6 are the best measures of this factor and 
they are the well-known Card Sorting and Card Assembly tests. 
The next best test appears to be test 9, which is the Machine 
Operator's Test described in the discussion of Factor II. Tests 
which have correlations of approximately .30 or greater with 
this factor tend to contribute to its meaningfulness; variables 
4 and 7 are possible exceptions. There is no apparent reason 
why variable 4 should be expected to correlate with the factor 
of manual dexterity, and it is somewhat surprising although 
not wholly unreasonable to find a test of digit-symbol-sub- 
stitution highly correlated with the dexterity factor. The 
exact nature of the manual dexterity factor is not perfectly 
clear. To what degree excellence of performance on tests of 
this factor is due to a type of acuity of perception and recog- 
nition has not been determined. This question will be dis- 
cussed furthez’% it4 presentation of Factor V. Test 10, the 
Spatial Relations Test, which owes sixteen per cent of its vari- 
ance to this factor, obviously requires manipulative skill and 
would be expected to draw upon some sort of manual dexterity 
to a significant degree. 

Factor V is identified by tests which have been termed 
perceptual tests and have been found by previous investigation 
to determine a perceptual factor (9). 


Factor V—Perceptual Factor 
I II Ill IV V VI U2 


(7. Digit-Symbol) .......0.00 02 .03 03 .23 10 O1 58 
8. Letter Cancellation ......... Ol 00 .00 .06 57 00 36 
ll. Number Cancellation ...... 00 01 05 .07 yi 00 36 


The two cancellation tests, no. 8 and 11, have remarkably 
high saturations with this factor. Test no. 7, the Digit- 
Symbol Substitution Test, has unexpectedly low saturation. 
The reason for this test’s having a high saturation with the 
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dexterity factor rather than with the perceptual factor may 
actually be due to an undetected clerical or computational 
error. On the other hand, it is conceivable that the relations 
observed here are valid relations and perhaps some unrecog- 
nized aspect of the digit-symbol performance could, if under- 
stood, give insight into the nature of manual dexterity. It 
should be observed in passing that tests commonly employed 
in the measurement of perceptual ability, including cancella- 
tion tests such as appear in this battery, call for speed of 
simple, routine perception and the emphasis is not upon acuity 
of differential recognition and discrimination. Perhaps manual 
dexterity is a complex ability which does call for a differential 
recognition and discrimination which is not purely or primarily 
of a routine nature. The hypothesis that tests for manual 
dexterity draw upon two distinct abilities, a manipulative 
ability and a recognition ability, may be tested by determining 
the factorial composition of manipulative tests when given in 
a darkened room or under conditions to eliminate visual dis- 
crimination and comparing the results with the factorial com- 
position of these tests when they are administered under ordi- 
nary conditions. Factor analysis is not necessary to test this 
hypothesis, however, and other, simpler designs would un- 
doubtedly be more economical. 

Factor VI has low saturations. The relatively low inter- 
correlations among the tests which contribute to the steadi- 
ness factor are in part due to their unreliability. The tests 
with the highest loadings have certain common characteristics, 
and as a consequence the factor of steadiness has been 


hypothesized. 
Factor VI—Steadiness 
I II III IV V VI U2 


i see Bo ta oe 01 02 01 03 01 11 81 
4. Body Balancing ............ Ol .02 Ol Al Ol 10 74 
18. Steadiness, No. contacts .... Ol 02 Ol 02 01 14 79 
26. Tracing Board, No. errors .. .02 .00 00 08 00 21 69 
27. Tracing Paper, No. errors .. .00 Ol .00 .00 .00 16 83 


Body Balancing calls for the individual’s balancing himself 
on the ball of one foot while standing on a three-inch cube 
of wood. Buxton (2) reported a factor of manual steadiness 
based upon two tests, one of which was a thrusting test which 
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calls for an operation similar to the nine holes steadiness test. 
In a later paper, Seashore et al. (7) reported a steadiness factor 
which was based upon measurements of postural sway. The 
steadiness factor which is reported in this study is dependent 
upon a measure of postural steadiness, body balancing, as well 
as measures of manual steadiness. These data in combination 
with the data provided by other observers (4, 6, 8) suggest 
that there is one general steadiness factor (probably due to 
basically physiological individual differences conceivably in 
proprioceptive sensitivity or muscular tonus). The possi- 
bility that there exists only one general factor of steadiness 
has quite useful implications and the hypothesis may readily 
be tested. 

The conclusion is apparent from this analysis that no 
common factor could account for the intercorrelations among 
the various tests. On the contrary it is apparent that the 
inter-test correlations may be accounted for in terms of six 
independent and meaningful factors. Obviously such factors 
cannot be considered immutable or part of the universe, but 
among tests which the authors of the Minnesota study (as 
well as current investigators) consider to be important and 
representative, meaningful, well defined, independent, func- 
tional groupings have been demonstrated. 


Other Analyses of Mechanical Ability 


There are few factorial studies relating to mechanical 
ability reported in the literature; the most satisfactory one is 
Harrell’s, which he succinctly summarizes as follows: “The 
intercorrelations of thirty-seven variables, including the Min- 
nesota battery of ‘mechanical ability’ tests, the seven 
MacQuarrie tests of ‘mechanical ability,’ O’Connor’s Wiggly 
Blocks, and the Stenquist picture-matching test, were analyzed 
by Thurstone’s centroid method. Five factors, Perceptual, 
Verbal, Youth, Manual Agility, and Spatial, were taken out. 
Factors prominent in so-called mechanical ability tests are 
the Spatial and Perceptual ones with MacQuarrie’s dotting test 
significantly high in the Manual Agility factor. Each of the 
factors can be measured with group pencil-and-paper tests” 
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(3). Harrell’s Factor IV, agility, could readily be identified 
with the present writer's manual dexterity factor, IV, were it 
not for the participation of the dotting test in Harrell’s agility 
factor. Since there were no additional dotting tests in his 
battery or other tests of the ballistic type, we cannot be con- 
fident that the loading of the agility factor with this particular 
test has general significance. It may be suspected that this 
test is highly related with a “ballistic” or stereotyped move- 
ment factor; high correlations between MacQuarrie’s dotting 
and tapping tests have been reported (1). 

In “The Application of Multiple Factorial Methods to the 
Study of Motor Abilities” (2) Buxton found several factors 
which are in essential agreement with the results of the present 
study. His Factor I is a dexterity factor which appears to be 
identical with the dexterity factor described in the present 
study. His Factor III, a steadiness factor, is determined by 
manual tests only. Buxton’s Factor VI is found in neither the 
present study nor in Harrell’s study. It appears to call for 
general coordination of the muscles of the forearm, upper arm, 
and shoulder girdle. 

A “Multiple Factorial Analysis of Fine Motor Skills” by 
Seashore, Buxton, and McCollom has been reported (7). The 
results of this study have uncertain value for the broader prob- 
lem of mechanical ability for several reasons. The tests em- 
ployed are of a rather specific nature and may not in general 
be considered representative of tests for mechanical ability. 
The analysis was based upon a sample of 50 men, a small 
sample for factor analysis. Moreover, it appears likely that 
the authors might have extracted additional factors; from an 
examination of the factor matrix it does not appear that the 
zero-order correlations may be satisfactorily reproduced. As 
the writers themselves note, some of the factor loadings are 
inconsistent with the identifications of the tests and factors. 
Although the study was not intended to contribute directly to 
the problem of the nature of mechanical ability, it does support 
the implications of the present study by suggesting the exist- 
ence of a postural steadiness factor and a factor of stereotyped 
movements of the arm such as are called for by Factor III in 
the present study. 
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The literature offers several interesting studies of the or- 
ganization of strength, agility, and motor fitness. In a com- 
prehensive approach to the study of the nature of mechanical 
ability, cognizance must be given to such possible components. 
This aspect of mechanical ability will be treated in a forth- 
coming study. 

Discussion 

The occurrence of an orthogonal factor structure is of some 
interest because both Buxton and Harrell have found an 
oblique structure to be most appropriate in analyzing their 
variables. The orthogonal feature of the present analysis may 
be in part due to the age of the subjects. They are seventh- 
and eighth-grade boys; the subjects for the other analysis of 
Buxton and Harrell had been adults. Babcock and Emerson, 
in a study of MacQuarrie’s test for mechanical ability, have 
found intercorrelations among the subtests to increase in 
magnitude with age of subject (1). This interesting tendency 
is quite the reverse of that usually observed for the so-called 
mental tests. The tendency for mental abilities as defined by 
factor analysis to become increasingly independent of each 
other with increasing age and the apparent tendency of me- 
chanical abilities to become more highly related with each other 
with increasing age requires explanation, and it is possible 
that the final answer to this paradox may yield some insight 
into the nature of the respective classes of ability. Since all 
the mechanical tests employed in this study (and in other 
studies too) are not equally valid, it cannot be argued that 
excellence in practical mechanical work requires a high degree 
of all of these abilities and that therefore a type of automatic 
selection occurs which eventually results in those who have a 
high degree of all abilities being active in mechanical opera- 
tions and those deficient in any one being equally discouraged 
in all. It is not likely that any selective situation such as this 
can be invoked to explain this paradoxical trend. A slightly 
modified hypothesis may be offered, however; namely, that a 
high order of spatial ability is necessary for personally gratify- 
ing performance in any mechanical operation and that persons 
who are deficient in spatial ability may in general tend to 
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avoid mechanical work or participate with little zeal and as a 
consequence neglect the development of any purely manual 
facilities with which they might have started life. The ap- 
parent anachronism in the development of mechanical ability 
is provocative of speculation. Although the trend may not be 
verifiable its implications are important enough to warrant 
scrutiny of any suitable genetic records which may exist. 

It is of interest to compare the classification of the tests 
arbitrarily imposed by the authors at the beginning of the 
Minnesota investigation and the classification revealed by the 
empirical evidence offered by the intercorrelations and by 
subsequent factor analysis. The first arbitrary classification 
was Intelligence and this classification is validated by the 
results of the statistical analysis. The second classification, 
Simple Motor Tests, actually includes measures for two factors 
defined by the present factor analysis, i.e., Factors I and III, 
stereotyped movement and steadiness. The third group 
comprises the two balancing tests and is also included in the 
steadiness factor. The fourth group, Complex Eye-Hand 
Coordination Tests, comprises tests which determine Factor 
IV, identified in the present study as manual dexterity. The 
participation of the Digit-Symbol Substitution Test and of 
other measures in this factor suggests that the designation eye- 
hand coordination for this class of ability may be most appro- 
priate. As previously mentioned, however, further research 
is required before the exact nature of manual dexterity can 
be determined. It may well be that manual dexterity is an 
operational unit which when analyzed for logical content does 
demand a type of manipulative ability and visual discrimi- 
nation. It is equally possible, however, that manual dexterity 
or eye-hand coordination may be fractionated into two rela- 
tively independent components, visual discrimination and a 
manipulative ability. Group V, designated as Assembly Tests 
Involving Manipulation and Response to Spatial Relations, 
appears to be appropriately identified insofar as our analysis 
has revealed that these tests, though primarily dependent upon 
the spatial visualization ability, draw in varying degrees upon 
manual dexterity. This class probably was regarded as a 
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combination group, however; i.e., it is not apparent that the 
Minnesota authors foresaw that such tests as the Spatial 
Relations Test and the Assembly Test would be primarily 
determined by the same factors as the Paper Form Board Test. 
Demonstration that spatial ability is a primary component 
of such manual tests as the Stenquist Assembly as well as the 
Paper Form Board Test was probably first made by Harrell. 
The Stenquist Picture Tests, set aside in a separate class as 
requiring mechanical knowledge, are found actually to con- 
tribute to the spatial factor. Apparently the perceptual factor 
was entirely unanticipated by the Minnesota authors; the 
cancellation tests are classified under miscellaneous. 


Summary 


The present study comprises a review of the preliminary 
Minnesota experiment and a factorial analysis of the variables. 
The analysis reveals that interrelationships of the variables are 
of such a nature as to yield upon analysis six meaningful in- 
dependent factors. For the most part the loadings are ex- 
ceptionally high and the results of the analysis are excep- 
tionally definitive. The factors are: 


I—Spatial Visualization 
II—Stereotyped Movement 
I]I—Scholastic Ability 
I1V—Manual Dexterity 
V—Perceptual—Speed 
VI—Steadiness 
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SUGGESTIONS FOR THE CONSTRUCTION OF 
MULTIPLE-CHOICE TEST ITEMS 


CHARLES I. MOSIER, M. CLAIRE MYERS, ann HELEN G. PRICE 
State Technical Advisory Service, Social Security Board, Washington, D. C. 


IN wRITING items in a particular area it is possible, but not 
very profitable, to use the inspiration technique. One reads 
until he is inspired to write an item, jots it down, and then reads 
some more. Those concepts that fall readily into item form 
get tested over and over again; those more difficult to test go 
untested. This procedure is likely to result, among other 
things, in very spotty coverage of the subject-matter area. In 
writing items, as in other activities, planning is essential. 

The present paper presents three work tools for the con- 
struction of multiple-choice items: definition of the subject- 
matter area to be covered and systematic sampling of it; a 
check list of the kinds of questions which can be asked; and a 
summary of criteria for multiple-choice items. 

Although these materials were developed for use in ex- 
amining for public personnel selection, they are applicable, 
with minor changes, in the other fields of test construction. 

To cover a subject-matter area adequately, whether in 
educational testing, merit system examining, or aptitude test- 
ing, the first step should be a definition of the area or areas 
to be tested. Even in a limited field, all possible questions 
cannot be asked; it is necessary to resort to a sampling of the 
field. If, however, the individual’s performance on the sample 
is to represent his performance in the entire field, it is a truism 
that the sample must represent the field; the sample to be rep- 
resentative cannot be left to chance, but should be planned. 
The definition of the field states the limits which must be 
reached; it does not prohibit going beyond those limits. If 
items are written outside the boundary of the definition, they 
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can usually still be used; if no items are written on an area 
within the field, that part of the area has not been sampled. 
A picture may help to visualize this. 

The entire circle represents the particular subject-matter 
area, e.g., knowledge of elementary and intermediate statistics. 
Each subdivision represents a set of related concepts within 
the broader field. For example, J might be the concepts re- 
lating to frequency distributions; 2 those relating to central 
tendency; 3 those relating to dispersion, etc. Unless the total 
area is defined by correctly drawing the total circle, e.g., closing 
the circle in the lower right quadrant, the existence of the sixth 
subdivision escapes our attention. 


ia 


It is essential, if one is to test adequately for a knowledge 
of the principles and techniques in statistics, that he be pre- 
pared to include test items in each of the areas. It is not neces- 
sary to include one from every area in every examination; one 
should not, however, include three from section J and none 
from section 6 unless he is prepared to say that the concepts 
in J are very important and those in 6 are inappropriate to 
the purposes of the particular examination. 

After the area has been outlined, the next step is the identi- 
fication of its significant subdivisions. This can be accom- 
plished by listing the important concepts, topics, principles, or 
subdivisions of the field or of the various types of skills which 
contribute to the total. This list represents the specifications 
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for the set of items. It should not, generally, be a list of the 
items to be written, but of the sub-areas to be covered, with 
each area to be sampled by several items. Preparing such a 
list assumes either an authoritative outline already set up, or 
an over-all familiarity with the field. The chapter headings 
in a text or the paragraph headings within a chapter constitute 
one source of such a listing. Once made the list is important: 
first, as an index to the resulting assembly of items; second, as 
a guide to future item construction; and third, as a more 
elaborate definition of the types of items contained in the 
subject-matter area. Table 1, showing an outline of statistics, 
is presented as an illustration, not necessarily as a model. 








TABLE 1 
Outline of Statistics 
1. General 5. Dispersion 
Interpretations General 
Life Tables Sigma 


“Severity Rate” 

Simple computations 

Sources and bibliographies 
Tabular presentation of results 
Terms and symbols 

Planning surveys 

Compound interest formula 


2. Frequency Distributions 


Class intervals and limits 
Bimodality 

Symmetry 

Skewness 

Frequency polygon 
Kurtosis 

Percentiles 

Normal curve properties 
Binomial expansion 


3. Charts, Graphs and Index Numbers 


Ogive 

Moving average 

Straight-line equation 

Semi-log paper 

Interpolation and extrapolation 
Pictograms and circle diagrams 
Gantt, Lorenz, ratio, and time 
charts 

Graphic computation 

Index numbers 


4. Central Tendency 


Harmonic Mean 
Geometric Mean 


Average deviation 
Quartile deviation 


6. Correlation 
Interpretation and use 
Scatter diagram 
Pearson 
Other coefficients 
Regression lines 
Partial and multiple 
Coefficient of alienation 
Standard error of estimate 


7. Non-Linear Regression 
Trend lines 
Population curves 
Eta and Blakeman’s test 

8. Sampling 
Methods: random, weighted 
Sampling errors 
Interpretation, S.E., and C.R. 
Combination of samples 
Probability theory 
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tested has been completed, appropriate sources or sections of | 
the source should be used in constructing the items. 


being tested and the kind of question asked to test that con- 
cept. A further distinction which should be made is between 
the kind of question asked (e.g., what, why, who?) and the 
form of the test item used to ask the question (e.g., true-false, 
multiple-choice, completion). We are concerned here pri- 
marily with multiple-choice items and the form of the test item 
will not be considered further. In Table 2 is a check list of 
some of the kinds of questions which may be asked together 
with illustrative examples based on the concept of central 











Once the list of concepts, principles, skills or topics to be 


It is useful to distinguish between the concept, or the skill, 
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TABLE 2 
Types of Questions 
1. Definition 
a. What means the same as... 
b. What conclusion can be drawn from . . ‘as 


. Which of the following statements expresses this concept in different terms? 


. What purpose is served by . 
. What principle is exemplified ‘by . 
. Why is this done... . 


. What is the most important reason for . . . 


. What is the cause of . 
. Under which of the following conditions is this true... . 





Example: The value which is determined by adding all of the scores and divid- 
ing by the number of cases is known in statistics as the: 
(1) arithmetic mean; 

(2) median; 

(3) mode; 

(4) harmonic mean; 

(5) average deviation. 


— 
~ 


2. Purpose 


Example: The mean is obtained for the purpose of providing: 
(1) a single number to represent a whole series of numbers; 
(2) the central point in a series; 

(3) a measure of group variability; 

(4) an indication of the most frequent response given; 

(5) an estimate of the relationship between two variables. 


3. Cause 


Example: From which of the following measures of central tendency will the 
sum of the deviations equal zero? 

(1) the mean; 

(2) the mode; 

(3) the median; 

(4) an arbitrary origin; 
(5) any measure of central tendency. 
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4. Effect 
What is the effect of . 


b. If this is done, what will happen? 


c. 


oop 


Which of the following should be done (to achieve a given purpose)? 

Example: The arithmetic mean of 55 cases is 83.0. If 3 of the cases, with values 
of 82, 115, and 130 are deleted from the data, the mean of the remaining 52 cases 
will be: (1) 81.50; (2) 77.05; (3) 83.00; (4) 84.50; (5) 94.08. 


5. Association 


What tends to occur in connection (temporal, causal or concomitant associa- 
tion) with . 

Example: If the distribution of scores is skewed positively, the mean will be: 

(1) lower than the median; 

(2) the same as the median; 

(3) higher than the median; 

(4) relatively unaffected; 

(5) the same as the mode. 


6. Recognition of Error 


Which of the following constitutes an error (with respect to a given situation) ? 

Example: The mean should not be used as the measure of central tendency 
when: 

(1) the distribution of scores is significantly skewed; 

(2) there are a large number of cases; 

(3) a non-technical report is to be prepared; 

(4) the data are continuous; 

(5) other statistical formulae are to be computed. 


7. Identification of Error 


. What kind of error is this? 
. What is the name of this error? 
. What recognized principle is violated? 


Example: In computing the mean of a distribution from grouped data, the sums 
of the deviations above and below the arbitrary origin were found to be 127 and 
189, respectively. The final value for the mean was in error. Of the following 
possibilities, that one which is most likely to have caused the error is that the 
computor: 

(1) failed to note the correct sign in adding the mean of the deviations to the 

assumed origin; 

(2) used an assumed mean higher than the true mean; 

(3) omitted some of the cases in tabulating the data; 

(4) divided by the wrong number of cases; 

(5) multiplied by the wrong class interval value. 


8. Evaluation 


What is the best evaluation of . . . (for a given purpose) and for what reason? 

Example: When the number of cases is small, e.g., less than 20, and the magni- 
tude of the values is likewise small, the use of an assumed mean in the computa- 
tion of the mean can best be evaluated as: 

(1) less efficient than computation from actual values; 

(2) likely to distort the value obtained by the introduction of a constant 

error; 

(3) more accurate than the use of actual values; 

(4) neither better nor worse than computation by other methods; 

(5) applicable only if the distribution is reasonably symmetrical. 


9. Difference 


What is the important difference between 
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Example: Of the following statements, that one which characterizes the essential 
difference between the mean and the median as measures of the central tendency 
of a distribution is that: 

(1) the magnitude of each score does not contribute proportionately to the 

computation of the median but does for the mean; 

(2) the median is a point whereas the mean is a distance; 

(3) the mean is less affected by extreme values; 

(4) the median is easier to compute; 

(5) the median is more generally used. 


10. Similarity 


What is the important similarity between .. . . 
Example: The mean and median are the same in that they are both measures of: 
(1) central tendency; 
(2) distance; 
(3) position; 
(4) variation; 
(5) relationship. 
11. Arrangement 


In the proper order, (to achieve a given purpose or to follow a given rule) which 
of the following comes first (or last or follows a given item) 

Example: In computing the mean for data already grouped in class intervals 
the most efficient first step is to: 

(1) determine the arbitrary origin and enter the deviation values; 

(2) find the midpoints of the class intervals; 

(3) multiply the frequency in each interval by the midpoint of the interval; 

(4) add the column of scores; 

(5) find the reciprocal of the total number of cases. 


12. Incomplete Arrangement 


In the proper order, which of the following should be inserted here to complete 
the series? 

Example: In deriving the formula for computing the mean from grouped data 
using an arbitrary origin the following steps were taken: 


a. x= 4, 

b. 2X =i=X’+NA; : 
EX _ ots ix’ 
N N 


The step which is 2 between steps (a) and (b) is: 
(1) solving (a) for X; 
(2) summing (a) over the N cases; 
(3) multiplying by i; 
(4) adding A to both terms of (a); 
(5) dividing by N. 


13. Common Principle 


All of the following items except one are related by a common principle: 


. What is the principle? 
. Which item does not belong? 
. Which of the following items should be substituted? 


Example: All except one of the following items (arithmetic mean, median, 
mode, and quartile) are measures of central tendency; of the following statistics, 
that one which could be substituted in the series for the item improperly included 
is the: 

(1) harmonic mean for quartile; 
(2) average deviation for mode; 
(3) range for quartile; 
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(4) standard deviation for quartile; 
(5) 50th percentile for median. 


14. Controversial Subjects 


Although not every one agrees on the desirability of —————, those who support 
its desirability do so primarily for the reason that: 

Example: Although not every one agrees that the mean is the best measure of 
central tendency, those who advocate its general use base their recommendation 
on the fact that the mean: 

(1) has the smallest sampling error; 

(2) is easiest to compute; 

(3) is most readily understood; 

(4) is the most typical score; 

(5) is not affected by extreme values. 





Frequently a concept can be tested by a variety of ques- 
tions; for some concepts only one kind of question is appro- 
priate; for still others, several are applicable, but one or two 
are clearly most appropriate. To secure adequate sampling 
in the construction of items on any concept, the concept should 
be checked against the list and as many items written as seem 
appropriate, each asking a different kind of question. By 
writing items which test knowledge of a concept through sev- 
eral kinds of questions, it is frequently possible to sample the 
area at several levels of difficulty, ranging from the simplest 
to the most difficult. 

The construction of a number of different items on each 
concept included in the outline of the subject-matter field pro- 
vides a more effective means of meeting the objective of those 
who feel that internal weighting is essential to give the appro- 
priate emphasis to the more important concepts. The pro- 
ponents of internal weighting argue that since certain concepts 
are more important, extra credit should be given for the ques- 
tions on those concepts. If the desirability of such extra 
credit is granted, the weighting is more easily and more reliably 
accomplished by including extra questions than by doubling 
the credits for a single question, thereby doubling the effect 
of measurement errors in the question. The availability of 
several different questions on each concept makes it possible 
to include more than one for any that are considered more 
important. These considerations apply whether items are 
being constructed for a single examination or for a central file. 
The check list, Types of Questions, in Table 2 is not in- 
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TABLE 3 
Criteria for Constructing Multiple-Choice Items1 








I. General Validity 
1. Can we readily predict that those candidates who know the answer 
would, on the average, be better qualified for the purpose at hand than 
those who do not? 
2. Is the item thought-provoking, rather than calling merely for scraps of 
information? 


II. Item Content 

1. Is the content of the item important enough to justify a question—not 
so specialized that only a few highly selected experts could know the 
answer? 

2. Does the subject matter of the item appear to be reasonably related to 
some type of activity appropriately covered by the test? 

3. Are the subject matter and the phrasing of the item such that no emo- 
tional antagonism will be aroused on the part of the public and the candi- 
date? How would the item look in a newspaper unfriendly to the merit 
system? (Although this criterion is primarily formulated for merit 
system examiners, it has application in education testing; there is no need 
to antagonize students unduly in testing them.) 

4. Is the content of the item something which can be learned without hav- 
ing actually been on the job itself? 

5. Does the subject matter mirror a consensus of current authoritative be- 
liefs and opinions, rather than the opinion of one individual? 

6. Could general terms be used to make the item usable in more than one 
testing situation. If the principle is applicable only in one situation, has 
this fact been flagged by specific reference in the item? 

7. Does the item call for a knowledge of concepts, reasons, and relat:on- 
ships, rather than for mere factual information, whenever the former is 
appropriate? 

8. Could any part of the item, such as modifying phrases, qualifications, 

be omitted without significantly influencing the distribution of 
responses? 


III. Item Structure 
Premise 
Is a definite task clearly and unambiguously set, so that all candi- 
dates work on essentially the same problem? Is it sufficiently clear 
that an informed candidate could give the correct answer from the 
premise if it were written as a completion item with no choices 
given? 

2. Is the idea stated clearly and directly, with the answer an impor- 
tant part of the statement—not buried at the end of a preposition 
in a parenthetical “which clause”? 

3. Does the premise, combined with each choice, constitute a com- 
plete unit of thought, both ideologically and grammatically? 

4. Does the premise, combined with the right answer, constitute a 
definite and true concept? Is the premise phrased to require some- 
thing more (the correct choice) for completeness? 

5. If the question is stated negatively, can it be restated in positive 
terms? If not, has the attention of the candidate been directed to- 
toward the negative phrasing of the question? 

6. Is the premise phrased to ask for the best rather than the correct 
answer whenever possible? 

7. Is the premise stated so complexly that the item becomes a test of 
whatever is taken to understand a complex premise rather than a 
test of knowledge or reasoning? If so, is this what was intended? 





1 These criteria, stated in terms of merit system examining, can - made ai 
cable to other testing situations by appropriate changes in wording. 
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TABLE 3 
Criteria for Constructing Multiple-Choice Items 








Does the premise avoid all unnecessary content which might give 
away the answers to other items? 

If there are other answers conceivably as good as the intended 
answer, is the premise limited by the phrase, “of the following”? 


B. Correct Answer 


op 


Is there one and only one correct answer to the problem as set by 
the premise? If the intended answer is merely the best of those 
given, is this indicated clearly in the premise? 

Do competent authorities agree on the correct answer? 

Could the candidate distinguish the correct answer from the incor- 
rect answers without having read the premise? If so, the item is 
in need of revision. 

Does selecting the correct choice require a real or reasoned under- 
standing of the concept rather than mere recall or recognition? 

Is there no possibility that a candidate may select the correct re- 
sponse simply because it is the only one containing the same words 
or phraces as the premise or because of other external characteris- 
tics! 


C. Distracters 
1 


9. 
10. 


Are the distracters such that a person likely to be inferior on the 
job will think they are correct? Is each distracter so plausible that 
someone not knowing the correct answer will choose it? 

If there are popular misconceptions in the field, are the distracters 
designed to attract candidates holding those misconceptions? Can 
familiar or stereotyped phrasing in the distracters be used to make 
them sound plausible? Can words or phases similar to those in the 
premise be deliberately planted in the distracters to give them added 
plausibility? Can this increased plausibility be achieved without 
distracting too many of the qualified candidates as well? Are 
specific determiners, such as “always” or “never,” avoided? 

Are are distracters, as well as the intended answer, related to the 
premise, both ideologically and grammatically? 

Are there no distracters so nearly correct that well qualified persons 
are likely to accept them and be able to defend their answers? 
Can the question be answered by someone who knows, not that the 
intended answer is correct, but that the incorrect choices are 
obviously wrong? If so, does the item remain at the difficulty level 
originally intended? 

Do all the choices constitute possible answers to a single direct 
question implied in the premise? 

Are the distracters of about the same length and complexity as the 
correct answer? If not, are there at least two which are parallel with 
the intended answer? 

Do pronouns refer clearly to one and only one antecedent? 

Are the choices parallel in grammatical form and in meaning? 

Are there repetitions in the choices which could be avoided by 
putting the repeated thought into the premise? 


tended to be all-inclusive nor is it intended to prescribe the 


language to 


be used in asking any particular kind of question. 


Rather it is a guide to assist in formulating questions and a 
check to help insure that all of the appropriate kinds of ques- 
tions will be asked. 
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Once the concept to be tested and the kind of questions to 
be asked have been determined, there remains the task of 
framing objective questions which will accomplish the intended 
purpose. 

The task can be stated simply: To phrase a question in such 
terms that (a) all prospective examinees understand the task 
set; (b) those who have the requisite degree of knowledge will 
give the intended answer; and (c) all who do not will give 
another answer. Fashioning such an item is more difficult than 
formulating the problem. The criteria in Table 3 are set up 
as aids in constructing such items. They are by no means 
original; they have been assembled from various sources and 
are summarized here for convenience. The criteria in the 
table were formulated for use in merit system examining; each 
one, however, with slight changes in wording or in emphasis 
is applicable in other examining settings. These criteria should 
be applied to each item written; the item should not be con- 
sidered as complete until the writer is satisfied that it meets 
each criterion. 

After the concept has been identified, the question asked 
in a premise which is clear and the intended choice written, 
the next task is that of writing distracters. The ideal dis- 
tracters are those wrong answers actually given by repre- 
sentatives of the group for which the item is intended (not by 
some other group), in response to the question asked as a com- 
pletion item. Since item writers do not normally have ready 
access to such answers, it is up to them to project themselves 
into the situation and say, “If I were one of the group to whom 
this test is to be given and were asked this question as a com- 
pletion item and didn’t know the answer, what response would 
I give?” On the success with which the item writer can thus 
project himself into the situation of such candidates and pre- 
dict their responses rests the value of the item from a technical 
standpoint. 

In presenting a completed item the item writer is, in effect, 
saying: “This item is to be included in a test and given to a 
group of examinees, some qualified and some not qualified 
with respect to a particular objective. If we could accurately 
separate the qualified individuals from the unqualified, and 
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examine the responses of each group, we would find the keyed 
answer given by a relatively high proportion of the qualified 
group and by a low proportion of the unqualified. Each of the 
four distracters, on the other hand, should be given by a high 
proportion of the unqualified group and by a low proportion 
of the qualified.” If, after finishing an item, one cannot in all 
honesty make this prediction, the item should be worked on 
further. If this prediction can be made, it still needs empirical 
verification by observing the actual responses of the two 
groups, “qualified” and “unqualified,” as defined by an accept- 
able criterion. 

















INFLUENCE OF LEVEL OF MOTIVATION ON THE 
VALIDITY OF INTELLIGENCE TESTS 


L. D. HARTSON 
Oberlin College 


A ToTAL of 579 students entered Oberlin College during the 
period 1941-43 for whom IQ’s were available from tests taken 
previous to entering college. These data have been added to 
those for the group of 734 entering during the years 193440, 
concerning whom a report has previously been made (2), 
making a total of 1,313 students. These data have been 
studied for the purpose of checking the earlier results, and 
examining the relationship between aptitude and scholastic 
achievement at different levels of motivation. 


Follow-Up of Study Reported in 1941 


From these 1,313 students a total of 1,444 records are avail- 
able, since there are 131 cases for whom two IQ figures were 
reported. For some purposes these might have been averaged, 
but as that would have prevented their use for comparing the 
validity of the different varieties of test, each IQ has been 
treated as though it represented a different student. The same 
procedure being followed with those who became seniors, the 
records for 522 students yielded 577 IQ’s. The two populations 
employed in the study are, therefore, 1,444 and 577. The 
figures used in the computation of high-school scholarship rep- 
resent not the actual grades but “credit points” obtained by 
a system for equating different grading systems. College schol- 
arship is handled in centile scores, that for seniors representing 
the cumulative average for seven semesters. 

Correlations were computed (1) between IQ’s obtained from 
the following group tests: Otis, Terman, Henmon-Nelson, Na- 
tional, Kuhlmann-Anderson, from the individually adminis- 
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tered Stanford-Binet, and from the Ohio State University 
Psychological Examination, as one variable, and high-school 
scholarship, as one criterion; (2) between all of these tests and 
first-semester college scholarship, as a second criterion; (3) 
between the IQ’s on each of these test groups and the OSU Test 
scores. A comparison was also made (4) of the validities of 
the OSU Test scores and those obtained with each of the other 
tests, with scholarship used as the criterion at both the sec- 
ondary school and college levels. 


The Prediction of High-School and College 
Scholarship 


The results obtained with 1,444 students in general confirm 
those reported for the smaller group, as may be seen by com- 
paring Tables 1 and 2 with the corresponding tables in the 1941 
study. The group tested with the Henmon-Nelson Examina- 
tion again shows a higher validity than do the other tests 
yielding IQ’s (Table 1), even though this group yields a lower 
validity coefficient than do the students tested with the 
Kuhlmann-Anderson and with the Stanford-Binet, when their 
intelligence is measured by the OSU Examination (Table 2). 


TABLE 2 


Correlations between Various Tests and Scholarship 

















Scholarship OSU Test 
Test group N Hi 
gh- Ist-sem. 
school college Mean a 

en a ee 680 385 556 49.24 28.55 
NN ta iten dcp ne nascehewar 304 319 564 50.60 28.12 
Henmon-Nelson ......0.00ee00e: 258 477 569 48.68 25.32 
RE ee ae 48 399 621 55.08 26.21 
Kuhlmann-Anderson ..........+- 63 584 610 51.69 28.37 
Stanford-Binet .......ccccceccees 91 482 584 49.02 28.41 
OSU Test (Pre-Entrance) ...... 331 489 .640 49.46 29.14 





This fact offers some suggestion of superiority for the Henmon- 
Nelson Test, although this presumption is not supported by 
sufficient numbers to make it statistically substantial. 


The Mean IQ of the Oberlin Group 


Increase in the size of the population studied from 835 to 
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1,444 represents a change of one point in the mean IQ, or an 
increase from 121 to 122. The range is 90 to 169, 99 per cent 
exceeding the general mean of 100. Sigma is 10.7. Comparison 
with the Terman-Merrill sigma, 16.3 (3, p. 37), shows that 
variability among the Oberlin student-body is much more re- 
stricted than it is in the general population. The average IQ 
for the group who became seniors is the same as for the entire 
group of freshmen. 


Mean College Scholarship of Students at Different 
IQ Levels 


As the number of students who by 1943 had reached 
senior standing was more than twice the number for whom 
data were available three years earlier, 577 as compared to 
253, a report is again made of first- and seven-semester scholar- 


ship of students at different IQ levels (Table 3). The table 


TABLE 3 
Freshman and Senior Scholarship of Students at Different IQ Levels 




















Ist-semester college grades 7-semester college grades 

IQ N % Mean Range Possible Actual % Mean Range 
166-170 3 0.2 97.7 97-99 3 2 66.7 96.0 93-99 
161-165 1 0.1 48.0 48 0 
156-160 3 0.2 67.3 32-92 0 
151-155 8 06 854 45-98 2 1 50.0 87.0 87 
146-150 21 15 704 13-98 14 8 57.1 700 26-97 
141-145 36 2.5 65.8 4-99 16 10 62.5 69.1 12-99 
136-140 85 59 65.3 4-98 54 42 77.8 67.5 1-99 
131-135 117 8.1 598 1-99 64 49 76.6 50.2 5-99 
126-130 210 145 556 1-99 126 80 63.5 52.8 1-99 


121-125 290 20.1 53.5 1-99 189 123 65.1 $2.1 5-99 
116-120 294 204 443 1-99 190 118 62.1 42.2 1-94 


111-115 191 13.3 38.9 1-97 129 78 60.5 39.4 2-95 

106-110 123 84 33.6 1-97 84 44 52.4 32.9 2-69 

101-105 46 3.2 37.2 1-98 29 14 48.3 444 2-96 

96-100 9 06 28.0 3-42 9 5 556 31.0 26-36 

91- 95 6 0.4 26.5 7-69 6 3 50.0 18.3 6-35 
- 9 1 O.1- 27.0 27 0 

Total 1444 100.0 915 63.1 


577 





also shows the range of scholarship at each level, and the 
proportion of those in college long enough to reach senior status 
at each level. (Scholarship is indicated by centiles.) The 
table shows that: 
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a. Although the general tendency is for those of higher IQ 
to make the better scholastic records, the range of scholastic 
performance is remarkably similar at the different levels. 
Achievement ranges between the lowest and the highest decile 
at each level from an IQ of 101 to that of 141. One student 
with an IQ of 105 attained a 96 centile standing as a senior. 
That the IQ was, however, an inaccurate index of her ability 
is suggested by the fact that she made a 71 centile score on 
the OSU Test. 

b. Of 15 students with IQ’s above 150, 12 made exception- 
ally good records. 

c. Sufficient time had elapsed for 15 students whose IQ’s are 
below 101 to become seniors. Eight of these have obtained the 
A.B., but in only four cases was this achieved in the normal 
four-year period. 

d. Some degree of selective persistence is evidenced by 
comparing the upper with the lower half of the distribution. 
Of the 468 with IQ’s above 120, 315, or 67.3 per cent, persisted, 
as compared with 262, or 58.6 per cent, of the 447 below that 
point. The ratio of the difference in these proportions to the 
standard error of the difference is 2.74. 


Relationship between Aptitude and Achievement 
at Different Levels of Motivation 


It is a matter of common observation that one of the basic 
reasons why the correlations between scholastic aptitude test 
scores and scholarship is no higher is that the degree of moti- 
vation varies widely from student to student. After comparing 
several formulas for computing an index of effort, Fei Tsao (1) 
has proposed the one which follows: 

E 

FQ- Predicted E * 
in which “E” is an individual’s scholastic grade score, and 
“Predicted E” is the value of the scholastic score which might 
be predicted from his aptitude test score when one employs 
the ordinary regression equation. If a student’s grade average 
equals his predicted average, the value of the FQ is 100, this 
index representing normal effort; if his achievement is higher 


100 
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than his predicted achievement, the value of the FQ is more 
than 100; and vice versa. This formula was used in computing 
Effort Quotients for the 1,444 population, using the IQ and 
OSU Test scores as independent and comparable bases of cal- 
culation, along with the high-school scholarship, for the purpose 
of predicting first-semester college scholarship, and for the 577 
who had become seniors, calculating the FQ from the same two 
intelligence tests, but now employing first-semester scholarship 
for predicting scholarship for seven semesters. 


Correlations between Aptitude Test Scores and First- 


Semester Scholarship at Different FQ Levels 
Table 4 tabulates for each of the FQ levels the correlation 


TABLE 4 


Correlations at Different FQ Levels between Aptitude Test Scores and First-Semester 
Scholarship, when FQ is Computed from High-School Scholarship and (1) 
IQ, (2) OSU Test Scores. 

















r Scholarship centile . Scholarship centile 

FQ levels using using ~ 
IQ’s N Mean OSU N Mean 
130 & above .264 125 61.56 338 101 51.33 
120-129 438 198 64.39 610 155 56.68 
110-119 541 252 61.39 692 304 64.67 
100-109 465 269 49.12 642 292 50.34 
90- 99 244 223 42.85 483 199 42.34 
80- 89 373 134 43.51 549 164 46.77 
70- 79 279 95 28.89 469 97 29.22 
60- 69 245 75 34,87 518 51 37.55 
Below 60 145 73 18.97 .385 82 23.05 
Total 358 1444 50.00 566 1444 50.00 





between the aptitude test score and first-semester scholarship, 
together with the numbers and means, permitting a compari- 
son between the results when two different aptitude indices are 
used, the IQ and the OSU Test scores. The predictive index in 
this case is computed from high-school scholarship. 

These data indicate (1) that as the correlation between the 
IQ and first-semester grades is considerably lower than the 
corresponding correlation between scholarship and the OSU 
Test score (.358 as compared with .566) this therefore holds 
true at the different motivation levels. (2) There is consider- 
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able contrast between the size of r at the different levels. For 
the calculations employing the IQ, the range is from .145 to 
541; for calculations employing the OSU Test score, the range 
is from .385 to .692. (3) The level where the relationship be- 
tween test scores and scholarship is closest is 110-119. (4) 
The level where the students make the highest average grades 
is not that representing the highest FQ, but is one or two steps 
below the highest. (5) However, the lowest grades on the 
average are made by those with the lowest effort indices. 
(6) It is also at the level of low motivation where the correla- 
tion between aptitude and achievement is most out of step. 


Correlations between Aptitude Scores and Seven-Semester 


Scholarship at Different FQ Levels 
Table 5 is comparable to Table 4, except that in the calcula- 


TABLE 5 


Correlations at Different FQ Levels between Aptitude Test Scores and Seven- 
Semester Scholarships, when FQ is Computed from Ist-Semester Scholarship 
and (1) IQ, (2) OSU Test Scores; with Mean Scholarship and 


Percentage of Persistence 

















rt Scholarship centile r Scholarship centile 

FQ levels using using Percentage 

1Q’s N Mean OSU N Mean Persisting 
170-Over 592 76 72.61 A477 53 60.03 74 
150-169 618 79 69.93 .630 55 61.86 76 
130-149 601 82 61.84 .763 74 63.61 70 
110-129 645 68 52.12 .716 112 67.38 78 
90-109 .661 64 49.56 633 78 45.37 67 
70- 89 .244 64 33.63 512 75 34.83 60 
50- 69 .188 62 25.34 458 58 27.90 59 
30- 49 .005 49 18.56 .262 46 19.85 53 
Below 30 O15 33 11.56 .210 26 12.42 27 
Total .322 577 50.00 494 577 50.00 63 





tion of the FQ first-semester college scholarship, rather than 
high-school scholarship, is used in the prediction formula. Again 
a comparison is made of the correlations between aptitude test 
score and achievement (in this instance cumulative grades for 
seven semesters) at the different FQ levels. (1) the 7’s again 
run considerably higher where the computation of FQ is based 
on the OSU Test than when the IQ is employed. (2) The FQ 


extends over a wider range at both extremes than is the case 
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when predicting first-semester grades, reaching from below 30 
to over 170. (3) The range of r’s at the different motivation 
levels is even greater than was found in the case of the first- 





semester grades. Where the IQ is used it extends from .005 | 


to .661, and where the OSU Test score is used in computing 
the FQ, the range is from .210 to .763. (4) The correlation 


between aptitude and achievement is again closest at a point 


below the highest level, although it is nearer the upper limit | 
than is true in the case of freshmen. (5) Where the FQ is | 
based upon the IQ there is a perfect positive correlation be- | 
tween degree of motivation and mean cumulative scholarship, | 


but where the FQ is based upon the OSU Test score the highest | 


grades were made by those with motivation indices of 110-129. 


From this high point the grade curve slopes regularly in both | 


directions. (6) By the time the students with the lowest level 
of motivation reached senior status their grade average was a 
12 centile. The literal grade equivalent of a 12 centile average 


for seven-semester cumulative scholarship is a low “C,” whereas | 


the letter equivalent of the 73 centile is “B.” (7) Sufficient 
time had elapsed to make it possible for 915 out of the original 
freshman population of 1,444 to have become seniors. The last 


column in Table 5 indicates the proportion at each FQ level 


that persisted into the senior year—FQ being computed from 
OSU Test scores and first-semester scholarship. The largest 
proportion of those persisting occurred at the 110-129 level, 
where the per cent is 78. At the lowest motivation level the 
elimination is 73 per cent, and at the highest level it is 26 per 
cent. (The elimination represented by these figures is some- 


———EE 


what higher than normal; 38 men from the most recent classes | 
had left college to enter the armed forces. Their FQs are, | 


however, distributed over the population range in such chance 
fashion as not to disturb the general averages.) (8) For the 
total group the correlation between FQ and scholarship for 
seven semesters is .702, when FQ is computed from IQ, and 


559, when FQ is based on OSU Test scores. 
Relation between FQ and Phi Beta Kappa 


For the purpose of discovering the relationship between FQ 
and the highest level of scholastic attainment, Table 6 was pre- 
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TABLE 6 


Proportions at Decile Levels making Phi Beta Kappa when Students are Grouped 
according to 1) First-Semester Grades, 2) OSU Test Scores, 3) 1Q’s, and 4) 
FQ’s (derived from OSU Test Scores and Ist-Semester Scholarship). 








Ist-se- OSU 
Decile mester PBK © Test PBK 1Q’s PBK FQ’s PBK 
levels <i N %& “me N % N N % N N % 


/o 





91-100 59 41 69 47 316 G&G 2s 6 6 10 
81- 90 79 18 23 52 na G6 HH G&G ae 
71- 80 61 4 cj 54 iwDw se 3 67 ® 3 
61- 70 65 1 2 71 5 7 6 16 26 61 15+ 25 


51- 60 57 0 O 50 2 4 54 3 6 65 4 6 
41- 50 72 49 1 z ® 4 5 2 &s 3 
31- 40 62 61 41 1 2 62 
21- 30 56 67 1 : & 2 3 49 

11- 20 48 61 2 as ST 1 2 

1- 10 18 65 46 1 2 2 





pared to show the proportions elected to Phi Beta Kappa at 
different motivation levels, as compared with the proportions 
from equivalent decile levels of (1) first-semester scholarship, 
(2) OSU Test scores, and (3) IQ. Phi Beta Kappa election is 
usually granted to the upper eighth of the class. 

Table 6 shows that final scholastic honors can be best pre- 
dicted from freshman grades, 69 per cent having ranked in the 
highest tenth of the class as freshmen, whereas the correspond- 
ing figure for the OSU Test is 66 per cent, and for the IQ, 35 
percent. This might have been anticipated from the fact that 
the correlation between first-semester scholarship and cumu- 
lative seven-semester scholarship (which includes first-semes- 
ter grades) is .805, whereas the validity coefficient for the 
OSU Test is .559, and for the IQ it is but .322. PBK election 
was obtained by a few students who ranked as low as the lowest 
deciles in the test distribution. In terms of FQ, the level 
contributing the largest proportion to PBK membership is 
71-80, from which level 39 per cent were elected. This corre- 
sponds approximately to the 127-140 FQ group. 


Summary 


1. A total of 1,444 IQ scores are available for freshmen and 
577 for seniors, these having been derived from the following 
tests: Otis, Terman, Henmon-Nelson, National, Kuhlmann- 
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Anderson, and Stanford-Binet. Each student was tested with 
the OSU Psychological Examination. 

2. When the correlation figures are compared with those 
obtained with the smaller population, concerning whom a 
report was made in 1941, no significant differences appear. 

3. The mean IQ of the Oberlin group is 122, the range: 90- 
169, and sigma: 10.7. The mean for those who became seniors 
was also 122. 

4. Cumulative scholarship for seven semesters ranges from 
the lowest to the highest decile at each level from an IQ of 101 
to 141, and 8 students achieved the A.B. whose IQ’s were below 
101, although in half of these cases this required more than 
eight semesters. 

5. Some selective elimination on the basis of IQ is indicated 
by the fact that 67.3 per cent of those with IQ’s above 120 
became seniors, as compared with 58.6 per cent of those with 
IQ’s below 121. The comparable figures for the upper and 
lower half of the OSU Test distribution are 68.9 and 57.3 per 
cent. The FQ is a still better basis for predicting persistence, 
with 73.6 of the upper half as compared with but 52.5 per cent 
of the lower half continuing into the senior year. 

6. The correlation between aptitude test scores and fresh- 
man scholarship ranges from .145 to .541, when FQ is computed 
from IQ; from .385 to .692, when FQ is computed from OSU 
Test score. When the correlations are made with cumulative 
seven-semester scholarship, the range is still greater. Where 
the IQ is used it extends from .005 to .661; where OSU Test 
score is used it ranges from .210 to .763. 

7. Although the lowest correlations and lowest scholarship 
correspond to the lowest FQ’s, the highest correlations and 
highest achievement are found at levels somewhat below the 
highest FQ’s. 

8. It is the 127-140 FQ level which contributes the largest 
proportion of students elected to Phi Beta Kappa. 


Conclusions 


Data from this study confirm the results obtained in 1941 
with a smaller population to the effect that it is possible for 
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a student with an IQ as low as 91-95 to obtain an A.B. in 
Oberlin College. The fact that this achievement is apt to 
require more than the usual number of semesters emphasizes 
the role of determination and directionality. Analysis of the 
relationship between motivation and achievement shows, how- 
ever, that the highest grades are not usually made by those 
with the highest effort indices, but by those within the 56-85 
centile range. Because of the method by which the FQ index 
is computed, the student with an IQ of 100, who makes average 
grades, obtains a higher index than the one with an IQ of 170, 
whose achievement is commensurate with his ability; but the 
former is much less apt to achieve Phi Beta Kappa distinction. 
The study also shows that when students with low effort indices 
are grouped, the aptitude test scores give little or no basis for 
predicting their scholarship, whereas with an able group, who 
are well motivated, the correlation may be quite high. In the 
case of the OSU Test, the difference is represented by 1’s of 
.210 and .763. 
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STATISTICAL INTERPRETATION OF SYMPTOMS 
ILLUSTRATED WITH A FACTOR ANALYSIS 
OF PROBLEM CHECK LIST ITEMS 


STANLEY S. MARZOLF anp ARTHUR HOFF LARSEN 
Illinois State Normal University 


BEHAVIOR symptoms are often found to occur in conjunc- 
tion more frequently than can be accounted for by chance. 
When the behavior symptoms are of the maladjustive sort it 
is customary to refer to their conjunction as a syndrome. 
Though belief in such syndromes is supported by clinical ob- 
servation, this same sort of observation yields the conclusion 
that many cases of maladjustment cannot be considered typical 
of any recognized syndrome. Furthermore, observation reveals 
that it is difficult to specify an invariant conjunction of symp- 
toms by which a syndrome may be defined. Elsewhere (3) it 
has been suggested that these situations and related problems 
can best be clarified by a statistical interpretation of symptoms. 
One statistical conception which is useful is factor analysis. 

It is the purpose of this article to illustrate how syndromes 
may be conceived of in terms of factor analysis. Viewed in this 
way it will be seen that (1) atypical cases are not inconsistent 
with the recognition of syndromes, and (2) a syndrome is a 
“central tendency” concept which cannot be, and is not neces- 
sarily defined as, an invariant conjunction of symptoms. 

The Mooney Problem Check List (4) was given to 205 
upper classmen in Illinois State Normal University. This 
instrument consists of 330 items about which students might 
worry. In using the instrument the student is asked to under- 
line those items about which he worries. When this has been 
done, he is further instructed to circle those items about which 
he is seriously concerned. The ten most frequently underlined 
items were intercorrelated. This was done with the aid of 
285 
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computing diagrams (1) which provide an estimate of tetra- 
choric r. To have included more items would have involved 
cell entries so small as to make estimates grossly inaccurate. 
A direct centroid solution was made and this was transformed 
to orthogonal simple structure and to an oblique solution. 

The extent to which the ten items were found to intercor- 
relate is shown in Table 1. Since the reliability of the check 


TABLE 1 


Intercorrelations among the 10 Most Frequently Underlined Items of the 
Mooney Problem Check List 














hy 2. 3. 4. 5. 6. A 8. 9. 10. 

RS 7” 

2. 55 

2 38 38 “a 

4. 37 15 40 i 

. 28 08 40 .05 i 

6. .16 43 .24 .05 18 a 

7. 01 02 .00 15 18 .23 a 

8. 18 05 .03 i 15 .08 52 és 

9. .04 12 .08 .04 12 36 25 .28 - 
10. 25 .00 02 .04 .02 .07 .05 12 19 





* The items are as follows: 

1. Wondering if I'll be successful in life. 
. Wanting a more pleasing personality. 
. Lack of self-confidence. 
Afraid to speak up in class discussion. 
. Disliking financial dependence on family. 
. Taking things too seriously. 
. Not enough sleep. 
. Too little chance to read what I like. 
. Not enough time for recreation. 
10. Restless at delay in starting life work. 


© ONION Wm we bt 


list items is not known, but is probably low, all the intercorre- 
lations are very likely attenuated. The standard error of an rz, 
when the true correlation is zero, for an N of 205 is .069. 
Though the data are admittedly fallible, we do not believe 
that they are so much so as to prevent their use for the illus- 
trative purposes we intend. 

Though no negative correlations were obtained, it cannot 
be assumed that the true relationships are positive, owing to 
the unreliability of the obtained 7’s. 

The second factor residuals are shown in Table 2. 

The B-group procedure, recommended by Holzinger and 
Harman (2, p. 23), did not produce clear-cut groups, but it 
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TABLE 2 
Second Factor Residuals 











2. 3. 4. 5. 6. 7. 8. 9. 10. 
ig 096 -218 015 -.007 -112 -056  .038 -.038 .170 
2. .096 -.034 -.105 -133  .212 -071 -.092 .023 -.071 
3. -218 -.034 077,149 022, -.003  -.046 063 -.035 
4. 015 -.105 .077 -118 -123 074 .104 -040 -.017 
5. -007 -133 .149 -.118 003 .055 -.004 -.006 -.046 
6 -—112 212 022 -.123 .003 -027 -.196 .110 -.037 
7. -056 -.071 -.003 .074 055 -.027 134 -.130 ~-.085 
8. 038 -.092 -.046 .104 -004 -.196 .134 -.089 -.017 
9. -038 .023 .063 -.040 -006 .110 -130 -.089 .060 
10. 170 -.071 -.035 -.017 -.046 -.037 -.085 -.017 .060 





* Numbers refer to items in footnote to Table 1. 


was possible to arrange the items in a rough approximation of 
two groups. In Table 1 the first five items constitute one 
group and the last five a second. Since a group is defined as a 
set of items within a correlation matrix which correlate more 
with each other than with other items in the matrix, a group 
may be said to constitute a syndrome. How definitely the 
items of these groups should be thought of as a syndrome will 
be developed later after the results of the analysis are given. 
The fact that the items within a group show correlations with 
some items not in the group illustrates one reason why syn- 
dromes are not always clearly defined. If a greater number of 
items had been included in the matrix, the items which do not 
fit well in either of these two groups would probably be found 
to associate themselves with some of the added ones to consti- 
tute a third group or syndrome. 

In order to illustrate the applicability of factor analysis to 
the interpretation of symptom intercorrelations, we turn now 
to these data. A direct centroid factor solution was obtained. 
Only two factors seemed to be of sufficient reliability, as judged 
on the basis of the size of the residuals, to merit consideration. 
The centroid factors were transformed to orthogonal simple 
structure, and the result is shown in Table 3. 

By comparing the loadings of the items on the two factors, 
it will be seen that the factor axes pass through or very close 
to the points representing items three and seven (see Figure 1). 
In fact, the rotation from the centroid axes to orthogonal 
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TABLE 3 
Orthogonal Simple Structure 











Variable* M, M. Communality 

1. 797 101 645 

ra 552 .142 325 

3. 750 001 563 

4+ 431 118 .200 

5. 335 .197 151 

6. .290 405 .248 

7. .002 633 401 

8. 101 610 382 

9. .022 601 361 

10. .073 213 .050 
3.326 

Per cent of 


total variance 


19.0 14.3 33.3 





* Numbers refer to items in footnote to Table 1. 


simple structure was determined by these two items. This 
procedure follows Reyburn and Taylor (5), who criticize 
Thurstone’s criteria for obtaining simple structure. It is their 
contention that hypotheses concerning the probable nature of 
the factors should be used in determining the rotation, for to 
do so is just as defensible as to use hypotheses concerning the 
nature of factors as a basis for selecting items for intercorrela- 
tion. In the present instance, our choice was not one of com- 
plete freedom, for a meaningful analysis could not have been 
achieved by any arbitrarily chosen angle of rotation. 
Consideration of the factor structure as shown in Table 3 
indicates that items one to four are most closely related to the 
first factor and that items seven, eight, and nine are most 
closely related to the second factor. Thus one syndrome con- 
sists chiefly of confusion or worry about future success in life, 
a more pleasing personality, lack of self-confidence, and fear 
of speaking in class discussions. The second syndrome involves 
worry about lack of sleep, lack of opportunity to do preferred 
reading, and lack of opportunity to engage in recreation. 
These two syndromes will be recognized by counselors of college 
students as representing a sizeable proportion of those who 
come to their attention. The fact that the ten most frequently 
underlined items contain these two syndromes is thus in accord 
with expectation. 
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The three items, five, six, and ten, which are not so defi- 
nitely characteristic of the two syndromes, would probably 
align themselves with other symptoms if the correlation matrix 
included more items. This is most likely true for items five 
and ten, since the total variance attributable to these two 
factors is small. It is of course possible that with a larger 
correlation matrix the groups here identified might not exist. 
However, all clinical observation makes this possibility seem 
unlikely. The magnitude of the loadings would doubtless 
change, but it seems likely that two such clusters would be 
found and that it would be possible to pass axes through them 
without doing violence to the data. 

The distribution of the loadings on these two factors sug- 
gests the possibility that a bi-factor solution would be the best 
description of the data. However, since there were but two 
groups, such a solution was not made. It is doubtful if the 
solution, had it been made, would have added much to the 
interpretation, and at any rate the illustrative purpose of this 
discussion can be achieved without it. 

We shall now turn to the question of causal factors. The 
first syndrome contains one item, namely, worry about lacking 
self-confidence, which is apparently more basic than the others 
in that it serves to identify the causal factor at the root of the 
syndrome. The decision to consider this as the causal factor 
responsible for the first syndrome comes from psychological 
knowledge other than that presented in these data. It is an 
inference based largely on case studies. 

Inference from clinical observation is even more necessary 
if a possible causal factor responsible for the second syndrome 
is to be identified. Such observation suggests that the second 
syndrome may result from a lack of integration. Students 
are notoriously prone to run in all directions at once. A 
hierarchy of preferences is all too often absent with the result 
that students complain of not being able to study, read, or 
engage in recreation to the extent that they would like. 

The possibility of correlated factors leads to a considera- 
tion of an oblique solution. The correlated factor axes were 
located so as to pass near item one and through the cluster 
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formed by items seven, eight, and nine. The angle of separa- 
tion between these correlated reference vectors represents a 
correlation of .259. The location of the axes and their relation 
to the orthogonal reference vectors is shown in Figure 1. The 


M @ 











10 
| 
| 
| 
r) 
7 
| 9/6 
4 6 
/e 
Z . 
f “4. G 
Yo) ZF 
O 2 4 6 8 40 M, 
Ficure 1 
Orthogonal and Oblique Reference Vectors 
correlated factor structure and pattern are presented in Table 4, 


while the contribution of the two correlated factors is given 


















* Numbers refer to items in footnote to Table 1. 


in Table 5. 
TABLE 4 
Correlated Factor Solution 
Structure Pattern 
Variable* —K—K — 
ryG, ryGe G, G: 
A 803 .198 806 -.011 
2. 566 .208 549 .066 
3. 744 092 772 -.107 
4. 443 .169 .428 .058 
5. 360 236 320 153 
6. 344 438 .248 374 
4 .090 629 -.078 650 
8. 185 618 027 6ll 
9. .106 .600 -.053 614 
10. 102 .220 .208 
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The interpretation of the syndromes based upon the oblique 
solution does not differ greatly from that already discussed in 
relation to the orthogonal solution. There is, however, the 
problem of accounting for the correlation between these two 
correlated factors. 

Several plausible bases for such correlation may be sug- 
gested. Students doubtless differ in their likelihood of worry- 
ing about anything. They may also differ in their willingness 
to admit worry. Ability to identify the source of their dis- 
content may be another basis for differentiating students. 
Any one of these factors might be responsible for the correla- 
tion between the two vectors. Furthermore, lack of integra- 
tion may be a factor in producing feelings of inadequacy, and 
thus the factors would be correlated. While the basis for 
correlation cannot be identified, there are a number of reasons 
why it might be expected to exist. In fact, a general factor 
of adjustment-nonadjustment, and hence correlated factors, are 
more to be expected than are uncorrelated factors. 


TABLE 5 
Total Contribution of Correlated Factors 











Gi G2 
Gi 1,906 ere 
G: .027 1.397 


Grand Total 3.33 








Now let us suppose that all possible symptoms of malad- 
justment were intercorrelated and that all unreliability factors 
were absent. In this hypothetical situation, what might we 
expect factor analysis to reveal? It seems possible that a rela- 
tively small number of factors might be discovered, each rep- 
resenting an axis passing through a cluster of symptoms. 
These clusters would represent maladjustment syndromes. If 
symptoms of adjustment were also included in this hypothet- 
ical correlation matrix, we may conceive of bi-polar factors as 
representing the most adequate reference vectors. 

Now one of the principal values to be attained by a con- 
sideration of symptoms in the statistical framework of factor 
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analysis is the manner in which it enables us to deal with the 
typical and atypical cases. 

Consider first the orthogonal solution. Suppose that we 
have an individual with very marked feelings of inferiority 
or inadequacy, in other words one who “has” a large amount 
of the first factor. Since the second factor is uncorrelated 
with it, this individual may have much, none, or very little of 
the second factor. Assuming the factor loadings shown in 
Table 2 to be reliable, we would expect such a person to com- 
plain of worry about his future and the inadequacy of his per- 
sonality, to be lacking in self-confidence, and to fear to speak 
up in class. He would be typical of the first or inferiority 
syndrome. Another individual might have no inferiority feel- 
ings but be greatly lacking in integration. This person would 
be typical of the second or unorganized syndrome. What is 
most likely, however, is that relatively more individuals would 
have moderate amounts of both factors, that is, would feel 
moderately inferior and be moderately lacking in integration. 
Such persons would not be typical of either syndrome and 
would show symptoms present in both. Thus it appears that 
we might expect that most people would not be “typical” 
cases. There remains the rare possibility that a given indi- 
vidual, on the basis of probability, would have large amounts 
of both factors. This person would not be typical either. 

In the case of correlated factors the chief difference is that 
having a large amount of one factor would enhance the proba- 
bility of having a greater-than-average amount of the other, 
a circumstance which would make a clear-cut distinction 
between syndromes even less likely than in the case of 
uncorrelated factors. 

Factor analyses of intercorrelations between most fre- 
quently underlined items of the Mooney Problem Check List 
provide data for illustrative purposes. With these data the 
feasibility and desirability of factor analysis as a framework 
for clarifying clinical findings can be shown. It is demon- 
strated that though syndromes may be said to exist, it is not 
surprising that typical cases representing these syndromes are 
not more frequently encountered. Though these data deal 
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with the milder symptoms of maladjustment, the same prin- 
ciples may apply to the interpretation of psychoneuroses and 
psychoses. 

In addition to making the existence of the atypical case 
consistent with the concept of syndrome, it is also possible to 
understand why syndromes can scarcely be defined as an in- 
variant conjunction of symptoms. The magnitude of the cor- 
relations of the items of either of the syndromes with the factor 
assumed to be responsible for it is considerably less than per- 
fect. Therefore, even though the basic factor is present we 
cannot expect that all of the symptoms will be. Various con- 
tingencies of the environment may exist without any relation 
to the existence of the causal factor and may thus alter its 
manifestation. 

Furthermore, it is not inconsistent with the concept of 
syndrome for one symptom to be present in more than one 
syndrome; that is, to be moderately correlated with more than 
one causal factor. Item six, “taking things too seriously,” may 
be an example of such a possibility. 

A syndrome must be considered as a central tendency con- 
cept rather than an invariant conjunction of symptoms. Syn- 
dromes are recognized clinically because individuals with great 
amounts of the various causal factors do exist. These more 
dramatic instances are easily recognized. It is not inconsistent 
with their identification for us to find many other cases which 
are not typical. 

The data also lend support to clinical observations regard- 
ing the frequency of certain behavior patterns in students. 
The evidence is most certainly not conclusive. The statistician 
will be inclined to doubt its validity and the clinician may feel 
that such evidence is unnecessary. On this matter we can only 
endorse Wolfle’s (6, p. 40) statement that closer cooperation 
between clinician and factor analyst is desirable. 
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NEW TESTS OF 1944-1945 


California Test of Personality: Primary and Adult Series, by Louis P. Thorpe, Willis 
W. Clark and Ernest W. Tiegs, 1944. Scores are obtained for two main fields, 
self-adjustment and social adjustment. Each of these fields is further divided 
into six areas for which scores are obtained. Tests are in packages of 25, with a 
manual of directions. Primary series, Form A, per 25 tests, $1.00; Primary 
series, Form B, per 25 tests, $1.00; 5c per copy for tests when bought in smaller 
quantities; Primary specimen set, 25c. Adult series, Forms A and B, same prices 
as for Primary. Published by the California Test Bureau, 5916 Hollywood 
Blvd., Los Angeles 28, California. 





ERC Stenographic Aptitude Test, by Walter L. Deemer, Jr., 1944. Time, 50 min- 
utes. The purpose of this test is to estimate the probable performance ot pupils 
studying shorthand. There are 5 subjects: speed of writing, word discrimination, 
phonetic spelling, vocabulary, sentence dictation. Package of 25, $3.50; specimen 
set, 35c. Published by Science Research Associates, 228 S. Wabash Ave., Chi- 
cago 4, Illinois. 





Foust-Schorling Test of Functional Thinking in Mathematics, by Judson W. Foust 
and Ralph Schorling, 1945. Time, 45 minutes. This test is intended to measure 
the power to deal with mathematical relationships. It is usable in the 9th grade 
and above. Sold only in packages of 25 copies with manual and key at $1.35 
per package; specimen set, 15c. Published by World Book Company, Yonkers- 
on-the-Hudson, N. Y. 


General Clerical Test, 1944. Time, 50 minutes. This is a test which has been de- 
signed as a general and differential test for use in selecting persons for all types 
of clerical work. It is for high school and above. Sold in packages of 25 with 
manual and key. 1-3 packages, $3.25 each; 4-39 packages, $2.85 each; 40 or 
more packages, $2.60 each. Per copy when package is broken, 15c. Specimen 
set, 30c. Published by the New York Psychological Corporation, 522 Fifth Ave., 
New York 18, N. Y. 


Goldstein-Scheerer Test of Abstract and Concrete Thinking, by Kurt Goldstein and 
Martin Scheerer, 1945. These tests are for the use of psychologists, psychiatrists, 
and neurologists working with patients who have brain injuries. They are de- 
signed to measure both quantitatively and qualitatively the impairment of the 
function of the brain with reference to abstract and concrete reasoning. The 
monograph contains full instructions for the administration and interpretation 
of the tests. Three of the five tests are available: the Goldstein-Scheerer Cube 
Test, the Weigl-Goldstein-Scheerer Color-Form Sorting Test, and the Goldstein- 
Scheerer Stick Test. Cube Test: 2 design books, $4.75 per set; Kohs blocks, 
$1.00 per set of 12; record forms, sold only in packages of 50, $2.75. Stick Test: 
30 plastic sticks of 4 lengths, $2.00 per set; record forms, including mimeo- 
graphed supplement to the monograph, sold only in packages of 50, $2.75. 
Complete materials, all three tests, including a copy of the monograph, “Ab- 
stract and Concrete Behaviour,” $21.50. Published by the Psychological Cor- 
poration, 522 Fifth Avenue, New York 18, N. Y. 











How Supervise? by Quentin W. File. No time limit. Norms are given in percentile 
rank and standardized scores. This test is considered as a measuring instrument 
of the supervisors’ knowledge and insight concerning human relations in in- 
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dustry. It is designed to aid management in obtaining a clearer picture of its 
supervisors’ understanding of the more important general aspects of the job. 
There are 3 sections: Supervisory Practices, Company Policies, and Supervisor 
Opinions. Forms A and B. Sold in packages of 25 with manual and key. 1 to 
3 packages, $2.00 each; 4 to 39, $1.65 each; 40 or more, $1.50 each; single copies, 
10c. Specimen set, 30c. Published by the Psychological Corporation, 522 Fifth 
Avenue, New York 18, N. Y. 


Inventory of Factors GAMIN (Abridged Edition), by J. P. Guilford and H. G. | 
Martin. This test is designed to measure the following personality character- | 
istics: G, general pressure for overt activity; A, ascendency in social situations 
as opposed to submissiveness; M, masculinity of attitudes and interests as op- 
posed to femininity; I, lack of inferiority feelings; and N, lack of nervous tense- | 
ness and irritability. Supplement to the manual and scoring keys are to be 
printed shortly, 100 copies, $10.00; 50 copies, $6.00; specimen set without scoring 
keys, 25c; specimen set with scoring keys, 50c. Published by the Sheridan Sup- 
ply Company, P.O. Box 837, Beverly Hills, California. 
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Johnson Temperament Analysis, by Roswell H. Johnson, 1944. The form is self- 
administering and ordinarily requires 40 to 50 minutes. The following scores 
are obtained: nervous, depressive, active, cordial, sympathic, subjective, aggres- 
sive, critical, and self-mastery. Specimen sets include: test booklet, manual of 
directions, response record sheet, and 1 analysis profile sheet. Scoring stencils 
must be ordered separately. The price of the specimen set is 25c postpaid. Per 
25 tests, $1.25; 10c each in smaller quantities. Response record sheets, 14c 
each; analysis profiles, lc each. Published by the California Test Bureau, 5916 
Hollywood Blvd., Los Angeles 28, California. 
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Mellenbruch Aptitude Test for Men and Women, by Paul L. Mellenbruch, 1944. 
Time, 35 minutes. This test is designed to discover the degree and limits of the | 
mechanical trainability of men and women applicants for mechanical positions. | 
It is also intended as a counseling aid for selecting junior and senior high school | 
students who will profit from industrial arts training. Scoring key is included 
free with each order of 25 or more booklets. Published by Science Research 
Associates, 228 Wabash Ave., Chicago 4, Illinois. t 





Minnesota Multiphasic Personality Inventory, Group Form, by Starke R. Hathaway | 
and J. Charnley McKinley. Time required is from 30 to 90 minutes. The [| 
individual form, published earlier, has been adapted to group form for use with | 
those not requiring individual attention. For subjects from about 16 years of 
age upward. The following scores are obtained: hypochondriasis, depression, 
hysteria, psychopathic deviate, paranoia, masculinity, hypomania, validity, ten- 
dency to lie and tendency to give “cannot say” replies. Booklets may be reused. 
1-24 booklets, 25c each; 25 or more, 22c each. Keys (either hand-scoring or 
machine-scoring should be specified) anc manual, $7.50. I.B.M. answer sheets, 
$5.00 per 100; 6c each in smaller quantities. Published by the Psychological 
Corporation, 522 Fifth Avenue, New York 18, N. Y. 





Occupational Interest Inventory—Intermediate, Form A, devised by Edwin A. Lee 
and Louis P. Thorpe, 1944. For junior high-school students to adults. The fol- 
lowing scores are_ obtained: personal-social, natural, mechanical, business, the 
arts, the sciences, verbal, manipulative, computational, and interest level. Pack- 
age ‘of 25 tests with manual, $1.75; 8c each in smaller quantities; specimen set, 
25c. Published by the California Test Bureau, 5916 Hollywood Blvd., Los 
Angeles 28, California. 





Occupational Preference Inventory, by Paul P. and Ralph T. Brainard, 1945. No 
time limit. This blank replaces the Brainard and Stewart Specific Interest In- 
ventories, which will go out of print shortly. Occupational preferences are di- 
vided into 28 occupational sections, and the 28 sections are combined into 7 
major occupational fields: commercial, personal service, agriculture, mechanical, 
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professional, esthetic and scientific. Booklets can be reused. Inventory booklets 
with manual: 1 to 9 copies, 25c each; 10 to 99, 22c each; 100 or more, 20c each: 
Manual is 25c extra when fewer than 10 are ordered; record forms are sold only 
in packages of 50. 1 package is $3.75; from 2 to i9, $3.50 each; 20 or more, 
$3.00 each; per copy, 10c each. Specimen set is 50c. Published by the Psycho- 
logical Corporation, 522 Fifth Avenue, New York 18, N. Y. 


Personal Audit (Revised), Forms LL and SS, by Clifford R. Adams, 1945. No time 
limits. Form LL consists of 9 parts, each part containing 50 items. This form 
can be used with adults having the equivalent of a grammar school education or 
with senior high school students. The first six parts of Form LL are published 
separately as Form SS. Form SS is recommended for use with junior high 
school students or in other situations where testing time must be curtailed. 
Scoring requires no keys. The fields for which scores are obtained are: serious- 
ness-impulsiveness, tranquility-irritability, frankness-evasion, stability-instability, 
tolerance-intolerance, steadiness-emotionality, pease -fluctuation, content- 
ment-worry. Form LL: package of 25 tests with manual, $3.75; specimen set, 
35c. Form SS: package of 25 tests with manual, $2. 50; specimen set, 25c. 
Published by Science Research Associates, 228 S. Wabash Ave., Chicago 4, 


Illinois. 








Pintner General Ability Tests: Non-Language Series, by Rudolph Pintner, 1945. 
These group tests are designed to measure mental functions independently of 
word knowledge and facility, and may even be administered in pantomime with- 
out the use of language, using special directions which can be obtained on re- 
quest. The series is to consist of two batteries: the Intermediate Test for grades 
4 through 9, and the Advanced Test for grades 9 and above. The Intermediate 
Test is now available. Package of 25 tests with manual, $1.80; specimen set, 
si Published by the World Book Company, Yonkers-on-the-Hudson, New 

ork. 


P-L-S Journalism Test, by George H. Phillips, Harry Levinson and H. E. Schram- 
mel, 1944. Time, 40 minutes. This test is designed for use in high school and 
college classes which have completed a first course in journalism. Percentile 
norms are given. Per package of 25, directions and key included, $1.00 f.o.b. 
Emporia; $1.15 postpaid. In quantities less than 25, test 5c, directions 5c, speci- 
men set, 15c. Published and distributed by the Bureau of Educational Measure- 
ments, Kansas State Teachers College, Emporia, Kansas. 








Reading Achievement Test—Iniermediate Test: Forms A and B, by Donald D. Dur- 
rell and Helen Blair Sullivan, 1944. Time, 30 to 45 minutes. These forms of 
the Reading Achievement Test are designed for grades 3 to 6. Sold only in 
packages of 25 with manual and key. Form A or B, $1.55. Specimen set, 55c. 
Published by the World Book Company, Yonkers-on-the-Hudson, New York. 


Schrammel-Otterstrom Arithmetic Test, by H. E. Schrammel and Ruth Otterstrom, 
1945. Time, 50 minutes. This test consists of two divisions: Test II for grades 
4, 5, and 6; and Test II, for grades 7 and 8. Percentile norms are provided for 
interpreting the scores for each part and for the entire test, both mid-year and 
end-of-year testing. Per package of 25, directions and key included, 60c f.o.b. 
Emporia; 75c postpaid. In quantities less than 25; test 3c, key 3c, directions 5c. 
Specimen set, 20c. postpaid. Published and distributed by the Bureau of Edu- 
cational Measurements, Kansas State Teachers College, Emporia, Kansas. 





Stevens Reading Readiness Test, by Avis Coultas Stevens, 1944. These tests assist 
the teacher to group her beginning children for reading. They are not intended 
to displace intelligence or kindergarten tests. Package of 25 tests with manual 
and key, $1.80; specimen set, 20c. Published by the World Book Company, 
Yonkers-on-the- Hudson, New York. 





Survey of Mechanical Insight, designed by D. R. Miller, 1945. Time, 25 minutes. 
Designed to measure aptitude for solving the types of problems involved in jobs 











298 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


requiring the operation, maintenance, repair, or design of various types of ma- 
chinery. Package of 25 tests with manual and key, $2.50; 1lc each in smaller 
quantities. Published by the California Test Bureau, 5916 Hollywood Blvd., 
Los Angeles 28, California. 


Survey of Object Visualization, by D. R. Miller, 1945. Time, 20 minutes. This sur- 
vey requires the examinee to predict how an object will look when its shape and 
position are changed. Tentative percentile norms are given. The test is practi- 
cally self-administering and the scoring is completely objective. Package of 25 
tests, including a manual of directions and scoring key, $1.50; 7c a copy in 
smaller quantities. Published by the California Test Bureau, 5916 Hollywood 
Blvd., Los Angeles 28, California. 








Survey of Space Relations Ability, by Harry W. Case and Floyd Ruch, 1944. Time, 
15 minutes. This test is designed to measure the ability of the employee or 
applicant to perceive rapidly and accurately the relationships among objects in 
space. Tentative percentile norms are furnished. Package of 25 tests, includ- 
ing a manual of directions and scoring key, $2.00; specimen set, 25c. Published 
by the California Test Bureau, 5916 Hollywood Blvd., Los Angeles 28, Cali- 


fornia. 


Wechsler Memory Scale, by David Wechsler, 1945. This a short individual test 
of memory arranged for use by psychologists working in mental hospitals. Sold 
only in packages of 50 record forms, with a set of cards for one of the tests, 
$1.90; manual must be ordered separately, 45c; specimen set, including manual, 
design card and record form, 70c. Published by the Psychological Corporation, 
522 Fifth Avenue, New York 18, N. Y. 








Wellesley Spelling Scale, by Thelma G. Alper and Edith B. Mallory, 1944. This 
spelling ability test is for use at high school and college levels. Percentile norms 
are provided for the various grades from nine to college freshmen. Forms I and 
II, per 25 tests, 75c each; in smaller quantities, 4c per test. Published by the 
California Test Bureau, 5916 Hollywood Blvd., Los Angeles 28, California. 





The United States Armed Forces Institute Tests of General Educational Develop- 
ment, 1945. The purpose of these tests is to measure the extent to which all 
of the past educational experiences of the individual tested have contributed to 
his general educational development or to his ability to carry on an educational 
program in high school or in the first two years of college. Emphasis is placed 
on intellectual powers rather than detailed content. Cost of the tests may be 
had upon application to the distributors. Published by the American Council 
on Education. Distributed by the Cooperative Test Service of the American 
Council on Education, 15 Amsterdam Ave., New York 23, N. Y., and Science 
Research Associates, 228 S. Wabash Ave., Chicago 4, Illinois. 


High School Level Tests. These tests are to determine whether the individual 
has had the equivalent of a high school education. Standard scores are pro- 
vided; percentile norms for the total score are available. The tests include: 
Test 1. Correctness and Effectiveness of Expression. 

Test 2. Interpretation of Reading Materials in the Social Studies. 

Test 3. Interpretation of Reading Materials in the Natural Sciences. 

Test 4. Interpretation of Literary Materials 

Test 5. General Mathematical Ability. 


College Level Tests. These tests are to determine whether the individual is 
capable of carrying on advanced college work. Standard scores are provided; 
percentile norms are available. The tests include: 


Test 1. Correctness and Effectiveness of Expression. 

Test 2. Interpretation of Reading Materials in the Social Studies. 
Test 3. Interpretation of Reading Materials in the Natural Sciences. 
Test 4. Interpretation of Literary Materials. 
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Subject Tests. These tests are to determine the individual’s proficiency in spe- 
cific subjects on both high school and college levels. The tests include: 
Examination in English—High School Level; Book I: Reading and Interpreta- 
tion of Literature and Literary Acquaintance. Percentile norms for total score 
available for grades 10, 11 and 12. 

Examination in English—High School Level; Book II: Composition. Norms are 
available. 

Examination in English—College Level; Book I: Reading and Literary Ac- 
quaintance. Percentile norms for total score are available. 

Examination in English—College Level; Book II: Composition. Percentile 
norms are available. 

Examination in Business English—High School Level. No norms available. 
Examination in Commercial Correspondence—College Level. No norms avail- 
able. 

Examination in French Vocabulary—Lower Level. No norms available. 
Examination in French Reading Comprehension—Lower Level. No norms 
available. 

Examination in French Grammar—Lower Level. No norms available. 
Examination in French Vocabulary—Upper Level. No norms available. 
Examination in French Reading Comprehension—U pper Level: No norms avail- 
able. 

Examination in French Grammar—Upper Level. No norms available. 
Examination in German Vocabulary—Lower Level. No norms available. 
Examination in German Reading Comprehension—Lower Level. No norms 
available. 

Examination in Spanish Vocabulary—Lower Level. No norms available. 
Examination in Spanish Reading Comprehension—Lower Level—No norms 
available. 

Examination in Spanish Grammar. No norms available. 

Examination in Italian Vocabulary. No norms available. 

Examination in Italian Reading Comprehension—Lower Level. No norms avail- 
able. 

Examination in Italian Grammar—Lower Level. No norms available. 
Examination in Business Arithmetic—High School Level. No norms available. 
Examination in Advanced Arithmetic—High School Level. No norms available. 
Examination in Elementary Algebra—High School Level. Percentile norms for 
total score available. 

Examination in Second-year Algebra—High School Level. Percentile norms for 
total score available. 

Examination in Plane Geometry—High School Level. Percentile norms for total 
score available. 

Examination in Algebra—College Level. No norms available. 

Examination in Plane Trigonometry—College Level. No norms available. 
Examination in Analytic Geometry—College Level. No norms available. 
Examination in General Science—High School Level. No norms available. 
Examination in Biology—High School Level. No norms available. 

Examination in Physics—High School Level. No norms available. 

Examination in Chemistry—High School Level. Percentile norms for total score 
available. 

Examination in Meteorology—High School Level. No norms available. 
Examination in Senior Science—High School Level. No norms available. 
Examination in Chemistry—College Level. Percentile norms for total score 
available. 

Chemistry Examination in Qualitative Analysis. No scaled scores provided. 
Percentile norms available. 

Chemistry Examination in Quantitative Analysis. Percentile norms available. 
Examination in Organic Chemistry. Percentile norms available. 

Examination in Astronomy—College Level. No norms available. 

Examination in Elementary Psychology—College Level. No norms available. 
Examination in American History—High School a- | College Levels. Percentile 
norms available. 
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Examination in World History—High School Level. No norms available. 
Examination in Civics—High School Level. No norms available. 

Examination in Problems of Democracy—High School Level. No norms avail- 
able. 

Examination in German Grammar—Lower Level. No norms available. 
Examination in Business English—High School Level. No norms available. 
Examination in Commercial Correspondence—College Level. No norms avail- 
able. 

Examination in Business Arithmetic—High School Level. Percentile norms 
available. 

Examination in Gregg Shorthand—First Year—Secondary School. No norms 
available. 

Examination in Typewriting—First Year—Secondary School. No norms avail- 
able. 

a, in Typewriting—Second Y ear—Secondary School. No norms avail- 
able. 

Examination in Bookkeeping and Accounting—First Year—Secondary School. 
No norms available. 

Examination in Bookkeeping and Accounting—Second Y ear—Secondary School. 
No norms available. 

Examination in Mechanical Drawing—High School Level. No norms available. 
Examination in Engineering Drawing—College Level. No norms available. 
Examination in Machine Design—College Level. No norms available. 
Examination in Strength of Materials—College Level. No norms available. 
Examination in Surveying—College Level. No norms available. 
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Achard, F. H. and Clarke, Florence H. “You Can Measure the Probability of 

Success as a Supervisor.” Personnel, XXI (1945), 353-373. 

The authors studied 300 successful and indifferent supervisors in a public utility. 
They developed a supervisory rating scale which they found to be quite accurate and 
reliable for the four groups of supervisors studied. Successful supervisors had higher 
scores in tests of mental ability, personality and breadth of interest, and in ability to 
see and understand quickly (visual perception). The procedure proposed by the 
authors for the selection of supervisors includes: the selection of tests, the selection 
of candidates, sifting out of candidates on the basis of test scores, and final selection 
by executives, adding to ratings based on test scores (a) job knowledge, (b) versa- 
tility and adaptability, (c) mental, physical, and emotional fitness, (d) standing with 
colleagues, and (e) how well they “click” with prospective superiors. Elizabeth Bell. 





Allen, Mildred. “Relationship Between the Indices of Intelligence Derived from the 
Kuhlmann-Anderson Intelligence Tests for Grade I and the Same Tests for 
Grade IV.” Journal of Educational Psychology, XXXVI (1945), 252-256. 
The Kuhlmann-Anderson Intelligence Tests for Grade I were administered to 

300 school children in the middle of their first year. Near the beginning of their 
fourth grade they were given the same test for that level. Neither M.A., I.Q., nor 
Pc. Av. obtained at Grade IV were adequately predicted from the same scores in 
Grade I. The author believes the low relationship might be due to the differences 
in verbal content between the two tests (the first grade is all non-verbal) and that 
the results question the validity of long-range predictions from a group intelligence 
test at the non-verbal grade level. She concluded that a verbal group intelligence 
test has greater validity for predicting successful achievement in the tool subjects 
when given with an achievement test. This would also assist “in determining 
whether pupils are working up to their ability.” Elizabeth Beil. 





Altus, William D. “Racial and Bi-Lingual Group Differences in Predictability and 
in Mean Aptitude Test Scores in an Army Special Training Center.” Psycho- 
logical Bulletin, XLII (1945), 310-320. 

The dichotomous disposition of four bi-lingual groups, American Indian, 
Mexican, Filipino, and Chinese, permitted validating testing devices by the bi-serial 
correlation method. The highest average bi-serial correlations between four verbal 
subtests of the Wechsler Mental Ability Scale, Form B, and discharge versus gradua- 
tion, were found in the Negro group, including three in the .50’s. The Indian was 
next, and the Mexican lowest. The Filipino would have been second in average 
validity but for a low arithmetic subtest correlation. Verbal tests gave lower scores 
to the bi-lingual groups, but did not lose in validity. In mean test scores whites 
were definitely superior, Negroes were second, and Mexicans third. There was little 
difference between Chinese and Filipino, while the Indian was significantly lowest. 
The data prove nothing as to innate racial differences because of divergent back- 
grounds. Elizabeth Beil. 





Glick, H. N., Flynn, Elizabeth, and Macomber, Lois. “Some Comparisons Between 
the Original and the Revised Stanford-Binet Scales.” Journal of Educational 
Psychology, XXXVI (1945), 177-183. 

This article presents results of two studies conducted at Massachusetts State 

College with respect to certain comparisons between the original and revised Stan- 
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ford-Binet scales. The most significant finding was that the two scales measured 
practically the same thing. The new scale was found to be more valid as a measure 
of intelligence in testing slightly superior children. Also it was found that the new 
scale tested slightly higher and showed greater dispersion of I.Q.’s than the old. Jane 
Nathler. 





Goldfarb, William. “Note on a Revised Block Design Test as a Measure of Abstract 
Performance.” Journal of Educational Psychology, XXXVI (1945), 247-251. 
The Revised Block Design Test is an adaptation of the Wechsler Block Design 

Test. The adaptation involved two changes. The first was an extension of the 

time limit to five minutes for each card. The second was the introduction of a new 

scoring method whereby the subject received anywhere from zero to three credits per 
picture depending on the degree of accuracy. The test, consisting of seven designs, 
was administered to thirty adolescents with a mean age of 12.3. Each child was rated 
on the Revised Block Design Test, the Wechsler Block Design Test, the Wechsler 

Similarities Test, the Vigotsky Test, and the Weigl Test. All correlations were sig- 

nificant at the one per cent level. The correlation between the block design tests 

was .90. Both block tests offer a good measure of the ability to generalize. The 

Revised Block Design Test has a slightly higher correlation with other criteria of 

abstract ability. Betty Steele. 





Lewis, W. Drayton. “The Relative Intellectual Achievement of Mentally Gifted 
and Retarded Children.” Journal of Experimental Education, XIII (1944), 
98-109. 

The purpose of the investigation was to determine the degree of relative achieve- 
ment of mentally retarded and mentally gifted children. The subjects consisted of 
children from four hundred and fifty-five schools from thirty-six states. The Kuhl- 
mann-Anderson and the Unit Scales of Attainment Tests were given to twelve hundred 
subjects for each grade. Reading, geography, arithmetic, and language tests were se- 
lected from a battery of eleven tests. The basic assumption was that if either group 
were working up to capacity, the median achievement scores should diverge as much 
from the norms as do the median mental ages. The results show that the superior 
group fails to achieve expectations, whereas the retarded group achieves more than 
expectations. The author concludes that relative achievement, based solely on mental 
age placements, is not significant. The important factors may be chronological age 
and length of attendance in school. Betty Steele. 





London, Ivan D. “Psychology and Heisenberg’s Principle of Indeterminacy.” 

Psychological Review, LIT (1945), 162-168. 

The purpose of this article is to demonstrate that Heisenberg’s principle of 
indeterminacy is inapplicable for psychology. Statisticians in psychology are not 
justified in presenting this principle as a sufficient reason for the inability to eliminate 
or evaluate interference. The principle results from the attempt to extrapolate into 
the microcosm the same dual concepts of space and time attributed to macrocosmic 
events. The principle applies only to the sub-atomic world. Interferences in 
psychological situations are the results of methodological difficulties inherent in the 
situation and not because of Heisenberg’s principle. There are two types of 
phenomena in psychology: (1) Convergent phenomena in which behavior is deter- 
mined from the average behavior of its parts; (2) Divergent phenomena in which a 
single discontinuous event affects the whole aggregate. In the latter phenomena 
the indeterminacy deriving from the unpredictable behavior of a single electron 
deals with isolated and unique electrons which cannot be duplicated. Such in- 
determinacy cannot be made the basis of a systematic psychology. Betty Steele. 





Montagu, M. F. Ashley. “Intelligence of Northern Negroes and Southern Whites 
in the First World War.” American Journal of Psychology, LVIII (1945), 
161-188. 

Scores made on the intelligence tests administered by the U. 8. Army to Negro 
and white draftees during World War I are analyzed and discussed. Whites in any 
given state scored higher than Negroes in the same state on all tests and in all states 
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except Kentucky and Ohio, where the Negroes excelled on the Beta tests. However, 
it was also found that the median score made by Negroes in a number of states was 
better than that made by whites in several other states. The conclusion, supported 
by geographical evidence, is drawn that the lower scores of the whites, in the instances 
where they appear, are due to inferior socio-economic conditions, and the generally 
lower scores of the Negroes are similarly explained. Findings indicate that differences 
in performance on the tests between Negroes and whites are due solely to the action 
upon native endowment of differences in socio-economic history. Frances E. Smith. 





Munroe, Ruth. “Three Diagnostic Methods Applied to Sally.” Journal of Abnormal 

and Social Psychology, XL (1945), 215-227. 

Sally is the fictitious name applied to a normal, fairly well adjusted, Sarah Law- 
rence College-student, one of a group of eleven who were selected for an intensive 
study to determine with what accuracy certain testing procedures could portray the 
wholeness of personality and educative potentialities of the students. The Rorschach 
test, modified for large-scale use, was administered by Munroe, a graphological 
analysis was effected by Lewinson, ‘and Waehmer made an appraisal from spontaneous 
drawings. These findings were augmented by reports from the faculty adviser and 
instructors. The experimenters and teachers worked blindly and independently of 
each other. There was a sufficient degree of conformity between the reports, the 
analysis of the tests, and the accomplishments of the students to suggest that remedial 
methods indicated by the tests might have been of value. Helen Heath. 





Patterson, R. Melcher. “Evaluation of a Prolonged Pre-Academic Program for High 

Grade Mentally Deficient Children in Terms of Subsequent Progress.” Journal 

of Experimental Education, XIII (1944), 86-89. 

The Prolonged Pre-Academic Program is an experimental educational unit at 
the Wayne County Training School. On the basis of experimental data it was 
found that mentally deficient children of I.Q. levels 60-79 inclusive, educated on 
the time schedule described, appeared to progress in academic work when it was 
introduced at a rate which enabled them to arrive at approximately the same aver- 
age grade level by 16 years of age as they would have achieved had academic 
instruction started at the earliest possible mental age and been given continuously. 
Jane Nathler. 





Rabin, Albert I. “Psychometric Trends in Senility and Psychoses of the Senium.” 

Journal of General Psychology, XXXII (1945), 149-162. 

The Wechsler-Bellevue intelligence scale was administered to 150 individuals 
aged 60-84, admitted to the New Hampshire State Hospital, and 100 of the result- 
ing records were divided into 4 diagnostic groups: Senile, C.A.S., Miscellaneous (not 
including organic), and Non-Psychotic. Analysis of results showed no intra-test 
organizational or structural differences between classes. Large positive deviations 
occurred in the 3 major verbal subtests, while practically all performance subtests 
showed negative deviations. When scores of each subject were correlated with age, 
only performance subtests showed statistically significant negative correlations, 
though 7’s for verbal subtests were also negative. Changes occurring in performance 
and timed subtests appear to be due to age levels rather than differential diagnosis, 
with inflexibility and perseverative tendency as the significant factors in lowered 
performance. Findings of this investigation fail to agree with Wechsler’s dichotomy 
of tests which “do not hold up with age,” and do not support the statement that 
schizophrenia resembles “premature aging.” Frances E. Smith. 





Ryan, Thomas Arthur. “Merit Rating Criticized.” Personnel. Journal, XXIV 

(1945), 6-15. 

The author suggests the use of his Inv ventory of Personnel as a simpler pro- 
cedure based on “few doubtful assumptions” than the merit rating, until new and 
different methods for evaluating personnel are produced through research. He 
criticizes the graphic rating scale of today as being based on false logic, since it 
secures ratings on distinct traits, then assigns a total score (“adding apples and 
carts”), thus substituting one trait for another. Also, there is no known way of 
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validating the weights given traits. The halo effect he believes so strong that a single 
careful rating of a man’s “over-all value” is probably better. Ratings cannot be 
compared or corrected for constant errors with confidence. He also states that studies 
have shown their reliability to be generally low. Elizabeth Bell. 





Sargent, Helen. “Projective Methods: Their Origins, Theory, and Application in 

Personality Research.” Psychological Bulletin, XLII (1945), 257-293. 

The theoretical and historical background of the projective methods is presented, 
and the methods are discussed with regard to their application, under the classifica- 
tions of materials, functional uses, techniques, and purposes. Fundamental experi- 
ments testing the efficacy of the methods have been few, and there is need for further 
research bearing on underlying assumptions and predictive value. Experiments thus 
far conducted demonstrate the basic mechanism of projection and claim some degree 
of predictive success for projective methods. Methodological problems include 
questions of reliability, validity, and standardization. Evaluation of the methods 
stresses the importance of an open-minded attitude toward theory revision and 
innovations in method. Projective methods hold promise in clinical psychology 
for a science of diagnosis and treatment; there is considerable evidence that they 
yen thorough exploration. The bibliography contains 274 references. Frances 

. Smith. 





Smith, Francis F. “The Use of Previous Record in Estimating College Success.” 

Journal of Educational Psychology, XXXVI (1945), 167-175. 

The purpose of the investigation was to predict, on the basis of high-school 
records, the number of grade-point scores each freshman would earn in his first semes- 
ter in college. The high-school record, based on about eighty examinations evaluated 
by sixteen instructors at different times, was converted into one figure indicating the 
percentage of recommended grades. A multiple-prediction formula was used which 
included the freshman’s aptitude percentile, reading percentile, and English A 
percentile. The subjects were 903 freshman students. The standard error of estimate 
from the multiple-prediction formula was 7.97. The results show a correlation of 
.60 between the high-school record based on a single figure and actual success in the 
first semester in college. However, the score loses its predictive value after a year 
or more. It was found that the best indicator of future scholastic success was the 
previous semester’s record. Betty Steele. 





Swift, Joan W. “Reliability of Rorschach Scoring Categories with Pre-School 

Children.” Child Development, XV (1944), 207-216. 

Reliability for Rorschach scoring categories was tested with pre-school children 
as subjects under the following conditions: (1) Test-retest over a 30-day interval. 
Reliabilities ranged from +.15 to +.83. (2) bag over a 14-day interval with 
a second series of ink blots interpolated on the 7th day. Uncorrected r’s ranged 
from + .41 to +.74, and corrected values for scores obtained by combining test and 
retest from +.59 to +.84. (3) Use of a parallel series of ink blots over a 7-day 
interval, with subjects identical with those used in (2). The 7’s ranged from —.06 
to +.84, all but one being above +.39, while corrected values for scores obtained by 
combining the 2 series of cards ranged from —.08 to +.86. (4) Test-retest over a 
ten-month interval, with 20 subjects retested as part of (2) ten months after 
participation in (1). Range of r’s was from +.18 to +.53. Reliability was affected 
by small number of responses given by each child, low frequency of occurrence of 
many response categories, and variability of interest and attention span of the 
children. Frances E. Smith. 





Swift, Joan W. “Matchings of Teachers’ Description and Rorschach Analyses of 
Pre-School Children.” Child Development, XV (1944), 217-224. 
Investigation was made of the validity of the total personality picture given by 

the Rorschach method when applied to pre-school children. Validating material 

consisted of a 250-word personality sketch of each of 30 pre-school children, written 
by the children’s teachers. Of the thirty matchings, 14 were correct and 16 incorrect, 

a result found to be significantly better than chance at the 1 per cent level of confi- 
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dence. All but 3 of the 14 correct matchings were of boys, a result possibly due to 
sex differences in stereotypy of behavior. Frances E. Smith. 





Wells, F. L. “Mental Factors in Adjustment to Higher Education.” Journal of 

Consulting Psychology, IX (1945), 267-286. 

“An introduction to the psychometric work of the Grant Study, concerned with 
well-adjusted Harvard undergraduates,” with reference to the “technical problems 
in the measurement of the superior intellect, . . . career choice, as well as educational 
prognosis.” It is concluded that the abstraction of the liberal arts college curriculum 
is not the real problem, but the fact that it is offered to many who probably cannot 
assimilate it. “A more highly ideational type of mind than commonly receives it” 
profits most from it. One half of the present college population could be replaced 
from the ranks of those economically not so fortunate. Two broad mental qualities 
are needed for college scholastic success, absorptiveness and creativity. The latter 
is hard to recognize with our present methods of measurement. For this reason, 
objective tests are believed inadequate for higher education. Eight case studies of 
well-adjusted students are presented. Elizabeth Bell. 





Woodrow, Herbert. “Intelligence and Improvement in School Subjects.” Journal 

of Educational Psychology, XXXVI (1945), 155-166. 

Studies were conducted at Litchfield, Ill., and Providence, R. I., to determine 
whether improvement is synonymous with intelligence and whether there is any 
common factor in improvement. In the Litchfield study, the Metropolitan Achieve- 
ment Battery and the Otis Quick-Scoring Mental Ability Tests were given to fourth-, 
fifth-, and sixth-grade children in May of successive years. Intercorrelations of 
gains in six school subjects were .12. Factor analyses revealed no gain factor 
common to the six subjects. Three factors were found: 1) an intelligence factor 
determining the I.Q., but not the gain; 2) an arithmetic factor; 3) a reading and 
English factor. In the Providence study the Stanford Achievement Test was used. 
The differences in standard scores for periods of one, two, and three years revealed 
low gain correlations. Three factors were found which were similar to those found 
in the Litchfield study. The conclusion of both studies is that there is no general 
gain factor, and that there is no significant relation between change in score and I1.Q. 
Bette Steele. 
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